≡ Menu

Shell scripting

Shell Scripting Hack: cp Command With a Progress bar

This is a nice shell scripting hack. It allows you to give a progress bar like wget to cp command.

Rotate FTP Backup Using a Shell Script

I’ve already written about rotating sftp / ssh backup shell script to remove directories (old backup files). However, a few of our readers would like to know more about removing old backup directories using ftp. As usual, you need accurate date and time on local system and remote backup directory must be in dd-mm-yyyy or mm-dd-yyyy format. For example daily mysql backup should be stored in /mysql/dd-mm-yyyy format.

Sample Shell Script

Here is a simple and dirty shell script to remove old backups ( download link ):

# call ./script.sh 03-2007 - to remove all March-2007 directories in 01-03-2007, 02-03-2007, 31-03-2007 format
# you must have ncftp ftp client installed on BSD / Linux box
BASE="/mysql" # base dir below that dd-mm-yyyy
[ $# -eq 0 ] && exit 1 || :
echo "Getting old directories..."
ncftpls -u 'ftp-user-name' -p 'ftp-password' -x "-t" ftp://ftp.your-server.com${BASE} > /tmp/ftp.out
LIST="$(grep ${DELETE} /tmp/ftp.out)"
echo -n "Starting removal for ${DELETE}..."
for dir in $LIST
# echo "Processing ${dir}..."
 ncftp -L -u 'ftp-user-name' -p 'ftp-password' ftp.your-server.com <<EOF
 cd $rdir
 rm *
 rmdir $rdir

Run the script as follows to remove all backup for Dec-2007, enter:
$ ./script.sh 12-2007

Related: Generate backup ftp script using php based wizard

Linux Command Line List ( PDF Version )

This is easy to use Linux command line index. Linux commands divided into categories such as:
=> System information
=> Shutdown
=> Files and Directory
=> File search
=> Mounting a Filesystem
=> Disk Space
=> Users and Groups and others

Linux Commands Line list

Linux / UNIX Bash: Copy Set of Files to All Users Home Directory

If you would like to copy a set of files for all existing users, use the following scripting trick. It will save lots of manual work.
[click to continue…]

Get Free Interactive Shell Access to Linux / BSD and UNIX Like Operating Systems

From my mail bag:

Where can I get free interactive access to HP-UX or Linux distro or UNIX shell access?

You can simply grab and try out any Linux / BSD / Solaris Live CD. However, some time you cannot install and use particular UNIX like os. So, if you want to try the latest technologies over the Internet? Try HP TestDrive program:

This program allows you to testdrive some of the hottest hardware and operating systems available today. Have you ever wanted to try out HP’s exciting 64-bit Integrity and PA-RISC technology? Get time on SMP x86 and Opteron ProLiant servers? Try out a Blade server. Try different Open Source operating systems such as FreeBSD, Suse, Redhat, Debian and other Linux distributions.

This program is perfect for students and new users to try out and learn basis of UNIX. You can also try and test your C/C++ programs using latest Intel compilers. It is intended for those users who want to sample the 32- and 64-bit servers running a variety of HP, UNIX, Linux and third-party operating systems and applications.

=> HP Test Drive Program [hp.com]

sed tip: Remove / Delete All Leading Blank Spaces / Tabs ( whitespace ) From Each Line

The sed (Stream Editor) is very powerful tool. Each line of input is copied into a pattern space. You can run editing commands on each input line to delete or change the input. For example, delete lines containing word DVD, enter:
cat input.txt | sed ‘/DVD/d’

To Print the lines between each pair of words pen and pencil, inclusive, enter:
$ cat input.txt sed -e '/^PEN/,/^PENCIL/p'

To remove all blank lines, enter:
$ cat /etc/rssh.conf | sed '/^$/d' > /tmp/output.file

sed is very handy tool for editing and deleting unwanted stuff. Following echo statements printed lots of whitespace from left side:
echo "     This is a test"

         This is a test

To remove all whitespace (including tabs) from left to first word, enter:
echo "     This is a test" | sed -e 's/^[ \t]*//'

This is a test


  • s/ : Substitute command ~ replacement for pattern (^[ \t]*) on each addressed line
  • ^[ \t]* : Search pattern ( ^ – start of the line; [ \t]* match one or more blank spaces including tab)
  • // : Replace (delete) all matched pattern

Following sample script reads some data from text file and generate a formatted output. It delete all leading whitespace from front of each line so that text get aligned to left:

exec 3<&0
exec 0<$FILE
while read line
	url=$(echo "http://${DOMAIN}${line}")
        title="$(lynx -dump -source ${url} | grep '<title>' | awk -F '<title>' '{ print $2 }' | cut -d'<' -f1|sed 's/^[ \t]*//')"
        echo "<li>${title}</li>"
exec 0<&3

To delete trailing whitespace from end of each line, enter:
$ cat input.txt | sed 's/[ \t]*$//' > output.txt
Better remove all leading and trailing whitespace from end of each line:
$ cat input.txt | sed 's/^[ \t]*//;s/[ \t]*$//' > output.txt

Print / Select a paper size other than A4 when using lp command line utility

Under Linux / UNIX you use lp command to print files from command prompt. lp is quite useful when GUI is not installed on Linux box to print files. The lp command is simply a front end command that calls the lpr command with appropriate options. Its main use is to allow the running of precompiled binary programs and scripts that assume that the lp command is the official printing command.

Changing papa size is very easy under GUI environment. But how do you change paper size under command prompt?
By default lp print to A4 paper size. However sometime you need to print to different paper size from command prompt such as A3 or A5. To print to A3 size, enter:
$ lp -o media=A3 /path/to/file

  • -o media=size : Sets the page size to size. Most printers support at least the size names “a4”, “letter”, and “legal”.

Other useful examples

Print a double-sided legal document to a printer called “hpljf2”:
$ lp -d hpljf2 -o media=legal -o sides=two-sided-long-edge /path/to/file
Print an image across 4 pages using a printer called “epd2”:
$ lp -d epd2 -o scaling=200 filename
Print a text file with 12 characters per inch, 8 lines per inch, and a 1 inch left margin to a printer called “lpodc2”:
$ lp -d lpodc2 -o cpi=12 -o lpi=8 -o page-left=72 ~/info.txt

To know more about lp option, enter:
$ man lp

Please note that you need to configure print using CUPS configuration file /etc/cups/cupsd.conf or web based tool located at http://localhost:631/
Linux / UNIX CUPS HP printer at http://localhost:631/
(Fig 01: My CUPS Configuration, showing HP PhotoSmart Printer )