Rackspace Cloud Files: Upload Files Using Linux / Unix FTP Command

Requirements

1. ftp-cloudfs – Python based FTP interface to Rackspace Cloud Files and OpenStack Swift. This act as a proxy to Rackspace cloud. By default the ftp-cloudfs will listen on port 2021 and you can connect to this port using any standard ftp client to upload / download files or create containers.
2. pip command – Install ftp-cloudfs.
3. Rackspace API username and password.

Install ftp-cloudfs

Type the following command ( see how to setup and install pip ):
$ pip install ftp-cloudfs
Sample outputs:

Downloading/unpacking ftp-cloudfs
  Downloading ftp-cloudfs-0.18.tar.gz
  Running setup.py egg_info for package ftp-cloudfs
Downloading/unpacking pyftpdlib (from ftp-cloudfs)
  Downloading pyftpdlib-0.7.0.tar.gz (94kB): 94kB downloaded
  Running setup.py egg_info for package pyftpdlib
 
    You might want to install pysendfile module to speedup transfers:
    http://code.google.com/p/pysendfile/
Downloading/unpacking python-cloudfiles (from ftp-cloudfs)
  Downloading python-cloudfiles-1.7.10.tar.gz (397kB): 397kB downloaded
  Running setup.py egg_info for package python-cloudfiles
    warning: no previously-included files found matching 'python_cloudfiles.egg-info'
Downloading/unpacking python-daemon (from ftp-cloudfs)
  Downloading python-daemon-1.6.tar.gz (41kB): 41kB downloaded
  Running setup.py egg_info for package python-daemon
Downloading/unpacking python-memcached (from ftp-cloudfs)
  Downloading python-memcached-1.48.tar.gz (40kB): 40kB downloaded
  Running setup.py egg_info for package python-memcached
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./py_virtual/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg (from python-daemon->ftp-cloudfs)
Downloading/unpacking lockfile>=0.9 (from python-daemon->ftp-cloudfs)
  Downloading lockfile-0.9.1.tar.gz
  Running setup.py egg_info for package lockfile
Installing collected packages: ftp-cloudfs, pyftpdlib, python-cloudfiles, python-daemon, python-memcached, lockfile
  Running setup.py install for ftp-cloudfs
    changing mode of build/scripts-2.6/ftpcloudfs from 644 to 755
    changing mode of /home/vivek/py_virtual/bin/ftpcloudfs to 755
  Running setup.py install for pyftpdlib
 
    You might want to install pysendfile module to speedup transfers:
    http://code.google.com/p/pysendfile/
  Running setup.py install for python-cloudfiles
    warning: no previously-included files found matching 'python_cloudfiles.egg-info'
  Running setup.py install for python-daemon
  Running setup.py install for python-memcached
  Running setup.py install for lockfile
Successfully installed ftp-cloudfs pyftpdlib python-cloudfiles python-daemon python-memcached lockfile
Cleaning up...

You also need to install a Python interface to sendfile to speed up file operation:
$ pip install py-sendfile pysendfile
Sample outputs:

Downloading/unpacking py-sendfile
  Downloading py-sendfile-1.2.3.tar.gz
  Running setup.py egg_info for package py-sendfile
Downloading/unpacking pysendfile
  Downloading pysendfile-2.0.0.tar.gz
  Running setup.py egg_info for package pysendfile
Installing collected packages: py-sendfile, pysendfile
  Running setup.py install for py-sendfile
    building 'sendfile' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c sendfilemodule.c -o build/temp.linux-x86_64-2.6/sendfilemodule.o
    gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/sendfilemodule.o -o build/lib.linux-x86_64-2.6/sendfile.so
  Running setup.py install for pysendfile
    building 'sendfile' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c sendfilemodule.c -o build/temp.linux-x86_64-2.6/sendfilemodule.o
    gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/sendfilemodule.o -o build/lib.linux-x86_64-2.6/sendfile.so
Successfully installed py-sendfile pysendfile
Cleaning up...

How do I start ftp-cloudfs proxy server?

Simply type the following command:
$ ftpcloudfs
By default ftpcloudfs will listen on 127.0.0.1:2021. You can verify this with the following commands. Make sure ftpcloudfs is started and you can view it with the ps command:
$ ps aux | grep ftpcloudfs
Sample outputs:

vivek    32314  0.2  0.1  43992  9408 ?        S    22:28   0:00 /home/vivek/py_virtual/bin/python /home/vivek/py_virtual/bin/ftpcloudfs
vivek    32315  0.0  0.1  52188  9268 ?        S    22:28   0:00 /home/vivek/py_virtual/bin/python /home/vivek/py_virtual/bin/ftpcloudfs
vivek    32322  0.0  0.1  43988  8840 ?        S    22:28   0:00 /home/vivek/py_virtual/bin/python /home/vivek/py_virtual/bin/ftpcloudfs
vivek    32333  0.0  0.0   8996   836 pts/8    S+   22:28   0:00 grep --color ftpcloudfs

Use the netstat command to verify that port 2021 is opened by ftpcloudfs, run:
$ netstat -tulpn | grep :2021
Sample outputs:

(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 127.0.0.1:2021          0.0.0.0:*               LISTEN      32314/python 

How do I use ftp client to connect to the Rackspace cloud via ftpcloudfs proxy?

You need to connect to the ftp proxy IP 127.0.0.1 at 2021 port:

+-------------+       +------------------+           +-----------------------+
|  ftp client | --->  | ftpcloudfs proxy |     | Rackspace cloud files |
+-------------+       +------------------+           +-----------------------+
                      Proxy IP: 127.0.0.1
                      Proxy Port: 2021

Type the ftp command as follows:

ftp 127.0.0.1 2021

You can also use lftp command as follows:

lftp -u USERNAME -p 2021 127.0.0.1
lftp -u USERNAME,PASSWORD -p 2021 127.0.0.1
lftp -e 'ftp-command1; ftp-command2;exit;' -u USERNAME,PASSWORD -p 2021 127.0.0.1
lftp -e 'ls;exit;' -u USERNAME,PASSWORD -p 2021 127.0.0.1

Example – Sample session

Linux and Unix like operating systems comes with ftp client which allows a user to transfer files to and from a remote network site:

ftp 127.0.0.1 2021

Sample outputs:

Connected to 127.0.0.1.
220 Rackspace Cloud Files 0.18 using pyftpdlib 0.7.0 ready.
Name (127.0.0.1:vivek): nixcraft
500 Command "AUTH" not understood.
500 Command "AUTH" not understood.
SSL not available
331 Username ok, send password.
Password: HIIDENPASSWORD
230 Welcome nixcraft
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
200 Active data connection established.
125 Data connection already open. Transfer starting.
drwxr-xr-x 745 nixcraft nixcraft 42698582 Oct 29 17:08 images.cyberciti.biz
drwxr-xr-x 100 nixcraft nixcraft   497143 Oct 29 17:08 static
drwxr-xr-x 555 nixcraft nixcraft 22378320 Oct 29 17:08 pdfs
drwxr-xr-x   1 nixcraft nixcraft        0 Oct 29 17:08 videos
drwxr-xr-x   1 nixcraft nixcraft        0 Oct 29 17:08 wpuploads
drwxr-xr-x 1346 nixcraft nixcraft 67375928 Oct 29 17:08 images
226 Transfer complete.

See how to use ftp command to upload / download files for more information about ftp commands.

A note about GUI ftp clients

If you do not want to type the commands, than use a browser such as Firefox and type the following at url bar:

ftp://127.0.0.1:2021/
ftp://USERNAME:PASSWORD@127.0.0.1:2021/

You can also use any standard GUI based FTP client.

A note about ftp

You may want to use sftp-cloudfs which provides SFTP (secure ftp) interface to Rackspace Cloud Files and Open Stack Object Storage (Swift) as described here. Stay tuned for sftp-cloudfs tutorial.

How do I download files from cloud files?

You can use URL to access files in this container via the public internet via cdn url such as follows:

http://8173579aa04ea33f3314-2bd3e6c481b4a486fea8b805a638363d.r25.cf2.rackcdn.com/file.txt

Login to your rackspace cdn account to see CDN enabled url:

Fig.01: Finding Cloud Files Akamai CDN URL via control panel

Check out related media

This tutorial is also available in a video format:

References

• ftp-cloudfs
• Rackspace (cp link)
• Openstack