Tutorial details | |
---|---|
Difficulty | Intermediate (rss) |
Root privileges | Yes/ No |
Requirements | Rackspace cloud files account and pip. |
Time | N/A |
Requirements
- ftp-cloudfs – Python based FTP interface to Rackspace Cloud Files and OpenStack Swift. This act as a proxy to Rackspace cloud. By default the ftp-cloudfs will listen on port 2021 and you can connect to this port using any standard ftp client to upload / download files or create containers.
- pip command – Install ftp-cloudfs.
- Rackspace API username and password.
Install ftp-cloudfs
Type the following command ( see how to setup and install pip ):
$ pip install ftp-cloudfs
Sample outputs:
Downloading/unpacking ftp-cloudfs Downloading ftp-cloudfs-0.18.tar.gz Running setup.py egg_info for package ftp-cloudfs Downloading/unpacking pyftpdlib (from ftp-cloudfs) Downloading pyftpdlib-0.7.0.tar.gz (94kB): 94kB downloaded Running setup.py egg_info for package pyftpdlib You might want to install pysendfile module to speedup transfers: http://code.google.com/p/pysendfile/ Downloading/unpacking python-cloudfiles (from ftp-cloudfs) Downloading python-cloudfiles-1.7.10.tar.gz (397kB): 397kB downloaded Running setup.py egg_info for package python-cloudfiles warning: no previously-included files found matching 'python_cloudfiles.egg-info' Downloading/unpacking python-daemon (from ftp-cloudfs) Downloading python-daemon-1.6.tar.gz (41kB): 41kB downloaded Running setup.py egg_info for package python-daemon Downloading/unpacking python-memcached (from ftp-cloudfs) Downloading python-memcached-1.48.tar.gz (40kB): 40kB downloaded Running setup.py egg_info for package python-memcached Requirement already satisfied (use --upgrade to upgrade): setuptools in ./py_virtual/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg (from python-daemon->ftp-cloudfs) Downloading/unpacking lockfile>=0.9 (from python-daemon->ftp-cloudfs) Downloading lockfile-0.9.1.tar.gz Running setup.py egg_info for package lockfile Installing collected packages: ftp-cloudfs, pyftpdlib, python-cloudfiles, python-daemon, python-memcached, lockfile Running setup.py install for ftp-cloudfs changing mode of build/scripts-2.6/ftpcloudfs from 644 to 755 changing mode of /home/vivek/py_virtual/bin/ftpcloudfs to 755 Running setup.py install for pyftpdlib You might want to install pysendfile module to speedup transfers: http://code.google.com/p/pysendfile/ Running setup.py install for python-cloudfiles warning: no previously-included files found matching 'python_cloudfiles.egg-info' Running setup.py install for python-daemon Running setup.py install for python-memcached Running setup.py install for lockfile Successfully installed ftp-cloudfs pyftpdlib python-cloudfiles python-daemon python-memcached lockfile Cleaning up...
You also need to install a Python interface to sendfile to speed up file operation:
$ pip install py-sendfile pysendfile
Sample outputs:
Downloading/unpacking py-sendfile Downloading py-sendfile-1.2.3.tar.gz Running setup.py egg_info for package py-sendfile Downloading/unpacking pysendfile Downloading pysendfile-2.0.0.tar.gz Running setup.py egg_info for package pysendfile Installing collected packages: py-sendfile, pysendfile Running setup.py install for py-sendfile building 'sendfile' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c sendfilemodule.c -o build/temp.linux-x86_64-2.6/sendfilemodule.o gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/sendfilemodule.o -o build/lib.linux-x86_64-2.6/sendfile.so Running setup.py install for pysendfile building 'sendfile' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c sendfilemodule.c -o build/temp.linux-x86_64-2.6/sendfilemodule.o gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/sendfilemodule.o -o build/lib.linux-x86_64-2.6/sendfile.so Successfully installed py-sendfile pysendfile Cleaning up...
How do I start ftp-cloudfs proxy server?
Simply type the following command:
$ ftpcloudfs
By default ftpcloudfs will listen on 127.0.0.1:2021. You can verify this with the following commands. Make sure ftpcloudfs is started and you can view it with the ps command:
$ ps aux | grep ftpcloudfs
Sample outputs:
vivek 32314 0.2 0.1 43992 9408 ? S 22:28 0:00 /home/vivek/py_virtual/bin/python /home/vivek/py_virtual/bin/ftpcloudfs vivek 32315 0.0 0.1 52188 9268 ? S 22:28 0:00 /home/vivek/py_virtual/bin/python /home/vivek/py_virtual/bin/ftpcloudfs vivek 32322 0.0 0.1 43988 8840 ? S 22:28 0:00 /home/vivek/py_virtual/bin/python /home/vivek/py_virtual/bin/ftpcloudfs vivek 32333 0.0 0.0 8996 836 pts/8 S+ 22:28 0:00 grep --color ftpcloudfs
Use the netstat command to verify that port 2021 is opened by ftpcloudfs, run:
$ netstat -tulpn | grep :2021
Sample outputs:
(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 127.0.0.1:2021 0.0.0.0:* LISTEN 32314/python
How do I use ftp client to connect to the Rackspace cloud via ftpcloudfs proxy?
You need to connect to the ftp proxy IP 127.0.0.1 at 2021 port:
+-------------+ +------------------+ +-----------------------+ | ftp client | ---> | ftpcloudfs proxy | | Rackspace cloud files | +-------------+ +------------------+ +-----------------------+ Proxy IP: 127.0.0.1 Proxy Port: 2021
Type the ftp command as follows:
ftp 127.0.0.1 2021
You can also use lftp command as follows:
lftp -u USERNAME -p 2021 127.0.0.1 lftp -u USERNAME,PASSWORD -p 2021 127.0.0.1 lftp -e 'ftp-command1; ftp-command2;exit;' -u USERNAME,PASSWORD -p 2021 127.0.0.1 lftp -e 'ls;exit;' -u USERNAME,PASSWORD -p 2021 127.0.0.1
Example – Sample session
Linux and Unix like operating systems comes with ftp client which allows a user to transfer files to and from a remote network site:
ftp 127.0.0.1 2021
Sample outputs:
Connected to 127.0.0.1. 220 Rackspace Cloud Files 0.18 using pyftpdlib 0.7.0 ready. Name (127.0.0.1:vivek): nixcraft 500 Command "AUTH" not understood. 500 Command "AUTH" not understood. SSL not available 331 Username ok, send password. Password: HIIDENPASSWORD 230 Welcome nixcraft Remote system type is UNIX. Using binary mode to transfer files. ftp> ls 200 Active data connection established. 125 Data connection already open. Transfer starting. drwxr-xr-x 745 nixcraft nixcraft 42698582 Oct 29 17:08 images.cyberciti.biz drwxr-xr-x 100 nixcraft nixcraft 497143 Oct 29 17:08 static drwxr-xr-x 555 nixcraft nixcraft 22378320 Oct 29 17:08 pdfs drwxr-xr-x 1 nixcraft nixcraft 0 Oct 29 17:08 videos drwxr-xr-x 1 nixcraft nixcraft 0 Oct 29 17:08 wpuploads drwxr-xr-x 1346 nixcraft nixcraft 67375928 Oct 29 17:08 images 226 Transfer complete.
See how to use ftp command to upload / download files for more information about ftp commands.
A note about GUI ftp clients
If you do not want to type the commands, than use a browser such as Firefox and type the following at url bar:
ftp://127.0.0.1:2021/ ftp://USERNAME:PASSWORD@127.0.0.1:2021/
You can also use any standard GUI based FTP client.
A note about ftp
You may want to use sftp-cloudfs which provides SFTP (secure ftp) interface to Rackspace Cloud Files and Open Stack Object Storage (Swift) as described here. Stay tuned for sftp-cloudfs tutorial.
How do I download files from cloud files?
You can use URL to access files in this container via the public internet via cdn url such as follows:
http://8173579aa04ea33f3314-2bd3e6c481b4a486fea8b805a638363d.r25.cf2.rackcdn.com/file.txt
Login to your rackspace cdn account to see CDN enabled url:
Check out related media
This tutorial is also available in a video format:
References
- ftp-cloudfs
- Rackspace (cp link)
- Openstack
🐧 4 comments so far... add one ↓
Category | List of Unix and Linux commands |
---|---|
File Management | cat |
Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
Network Utilities | dig • host • ip • nmap |
OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
Package Manager | apk • apt |
Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
Searching | grep • whereis • which |
User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
Great tutorial. Can’t wait for sftp related tutorial.
Nice job! Also check out cloudfuse for some local mounting of the cloud files. It uses fuse hotness to mount the cloud files container in user space.
Thanks for the comment and link. I will check it out.
Any way to do this for hubic? ftp-cloudfs is looking for an authurl. I have no idea what to fill in there.