≡ Menu

Shell scripting

netstat command and shell pipe feature can be used to dig out more information about particular IP address connection. You can find out total established connections, closing connection, SYN and FIN bits and much more. You can also display summary statistics for each protocol using netstat.

This is useful to find out if your server is under attack or not. You can also list abusive IP address using this method.
# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n

      1 CLOSE_WAIT
      1 established)
      1 Foreign
      3 FIN_WAIT1
      3 LAST_ACK
     17 LISTEN
    154 FIN_WAIT2
    327 TIME_WAIT

Dig out more information about a specific ip address:
# netstat -nat |grep {IP-address} | awk '{print $6}' | sort | uniq -c | sort -n

      2 LAST_ACK
      2 LISTEN
      4 FIN_WAIT1
     91 TIME_WAIT
    130 FIN_WAIT2

Busy server can give out more information:
# netstat -nat |grep | awk '{print $6}' | sort | uniq -c | sort -n

  64 FIN_WAIT_1
  65 FIN_WAIT_2

Get List Of All Unique IP Address

To print list of all unique IP address connected to server, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq
To print total of all unique IP address, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq | wc -l


Find Out If Box is Under DoS Attack or Not

If you think your Linux box is under attack, print out a list of open connections on your box and sorts them by according to IP address, enter:
# netstat -atun | awk '{print $5}' | cut -d: -f1 | sed -e '/^$/d' |sort | uniq -c | sort -n


You can simply block all abusive IPs using iptables or just null route them.

Get Live View of TCP Connections

You can use tcptrack command to display the status of TCP connections that it sees on a given network interface. tcptrack monitors their state and displays information such as state, source/destination addresses and bandwidth usage in a sorted, updated list very much like the top command.

Display Summary Statistics for Each Protocol

Simply use netstat -s:
# netstat -s | less
# netstat -t -s | less
# netstat -u -s | less
# netstat -w -s | less
# netstat -s


    88354557 total packets received
    0 forwarded
    0 incoming packets discarded
    88104061 incoming packets delivered
    96037391 requests sent out
    13 outgoing packets dropped
    66 fragments dropped after timeout
    295 reassemblies required
    106 packets reassembled ok
    66 packet reassembles failed
    34 fragments failed
    18108 ICMP messages received
    58 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 7173
        timeout in transit: 472
        redirects: 353
        echo requests: 10096
    28977 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 18881
        echo replies: 10096
    1202226 active connections openings
    2706802 passive connection openings
    7394 failed connection attempts
    47018 connection resets received
    23 connections established
    87975383 segments received
    95235730 segments send out
    681174 segments retransmited
    2044 bad segments received.
    80805 resets sent
    92689 packets received
    14611 packets to unknown port received.
    0 packet receive errors
    96755 packets sent
    48452 invalid SYN cookies received
    7357 resets received for embryonic SYN_RECV sockets
    43 ICMP packets dropped because they were out-of-window
    5 ICMP packets dropped because socket was locked
    2672073 TCP sockets finished time wait in fast timer
    441 time wait sockets recycled by time stamp
    368562 delayed acks sent
    430 delayed acks further delayed because of locked socket
    Quick ack mode was activated 36127 times
    32318597 packets directly queued to recvmsg prequeue.
    741479256 packets directly received from backlog
    1502338990 packets directly received from prequeue
    18343750 packets header predicted
    10220683 packets header predicted and directly queued to user
    17516622 acknowledgments not containing data received
    36549771 predicted acknowledgments
    102672 times recovered from packet loss due to fast retransmit
    Detected reordering 1596 times using reno fast retransmit
    Detected reordering 1 times using time stamp
    8 congestion windows fully recovered
    32 congestion windows partially recovered using Hoe heuristic
    19 congestion windows recovered after partial ack
    0 TCP data loss events
    39951 timeouts after reno fast retransmit
    29653 timeouts in loss state
    197005 fast retransmits
    186937 retransmits in slow start
    131433 other TCP timeouts
    TCPRenoRecoveryFail: 20217
    147 times receiver scheduled too late for direct processing
    29010 connections reset due to unexpected data
    365 connections reset due to early user close
    6979 connections aborted due to timeout

Display Interface Table

You can easily display dropped and total transmitted packets with netstat for eth0:
# netstat --interfaces eth0

Kernel Interface table
eth0       1500   0  2040929      0      0      0  3850539      0      0      0 BMRU

Other netstat related articles / tips:

  1. Get Information about All Running Services Remotely
  2. Linux / UNIX Find Out What Program / Service is Listening on a Specific TCP Port

Read following man pages for the details:
$ man netstat
$ man cut
$ man awk
$ man sed
$ man grep

Updated for accuracy.

This is a nice shell scripting hack. It allows you to give a progress bar like wget to cp command.

Rotate FTP Backup Using a Shell Script

I've already written about rotating sftp / ssh backup shell script to remove directories (old backup files). However, a few of our readers would like to know more about removing old backup directories using ftp. As usual, you need accurate date and time on local system and remote backup directory must be in dd-mm-yyyy or mm-dd-yyyy format. For example daily mysql backup should be stored in /mysql/dd-mm-yyyy format.

Sample Shell Script

Here is a simple and dirty shell script to remove old backups ( download link ):

# call ./script.sh 03-2007 - to remove all March-2007 directories in 01-03-2007, 02-03-2007, 31-03-2007 format
# you must have ncftp ftp client installed on BSD / Linux box
BASE="/mysql" # base dir below that dd-mm-yyyy
[ $# -eq 0 ] && exit 1 || :
echo "Getting old directories..."
ncftpls -u 'ftp-user-name' -p 'ftp-password' -x "-t" ftp://ftp.your-server.com${BASE} > /tmp/ftp.out
LIST="$(grep ${DELETE} /tmp/ftp.out)"
echo -n "Starting removal for ${DELETE}..."
for dir in $LIST
# echo "Processing ${dir}..."
 ncftp -L -u 'ftp-user-name' -p 'ftp-password' ftp.your-server.com <<EOF
 cd $rdir
 rm *
 rmdir $rdir

Run the script as follows to remove all backup for Dec-2007, enter:
$ ./script.sh 12-2007

Related: Generate backup ftp script using php based wizard

Linux Command Line List ( PDF Version )

This is easy to use Linux command line index. Linux commands divided into categories such as:
=> System information
=> Shutdown
=> Files and Directory
=> File search
=> Mounting a Filesystem
=> Disk Space
=> Users and Groups and others

Linux Commands Line list

If you would like to copy a set of files for all existing users, use the following scripting trick. It will save lots of manual work.
[click to continue…]

From my mail bag:

Where can I get free interactive access to HP-UX or Linux distro or UNIX shell access?

You can simply grab and try out any Linux / BSD / Solaris Live CD. However, some time you cannot install and use particular UNIX like os. So, if you want to try the latest technologies over the Internet? Try HP TestDrive program:

This program allows you to testdrive some of the hottest hardware and operating systems available today. Have you ever wanted to try out HP's exciting 64-bit Integrity and PA-RISC technology? Get time on SMP x86 and Opteron ProLiant servers? Try out a Blade server. Try different Open Source operating systems such as FreeBSD, Suse, Redhat, Debian and other Linux distributions.

This program is perfect for students and new users to try out and learn basis of UNIX. You can also try and test your C/C++ programs using latest Intel compilers. It is intended for those users who want to sample the 32- and 64-bit servers running a variety of HP, UNIX, Linux and third-party operating systems and applications.

=> HP Test Drive Program [hp.com]

The sed (Stream Editor) is very powerful tool. Each line of input is copied into a pattern space. You can run editing commands on each input line to delete or change the input. For example, delete lines containing word DVD, enter:
cat input.txt | sed '/DVD/d'

To Print the lines between each pair of words pen and pencil, inclusive, enter:
$ cat input.txt sed -e '/^PEN/,/^PENCIL/p'

To remove all blank lines, enter:
$ cat /etc/rssh.conf | sed '/^$/d' > /tmp/output.file

sed is very handy tool for editing and deleting unwanted stuff. Following echo statements printed lots of whitespace from left side:
echo "     This is a test"

         This is a test

To remove all whitespace (including tabs) from left to first word, enter:
echo "     This is a test" | sed -e 's/^[ \t]*//'

This is a test


  • s/ : Substitute command ~ replacement for pattern (^[ \t]*) on each addressed line
  • ^[ \t]* : Search pattern ( ^ - start of the line; [ \t]* match one or more blank spaces including tab)
  • // : Replace (delete) all matched pattern

Following sample script reads some data from text file and generate a formatted output. It delete all leading whitespace from front of each line so that text get aligned to left:

exec 3<&0
exec 0<$FILE
while read line
	url=$(echo "http://${DOMAIN}${line}")
        title="$(lynx -dump -source ${url} | grep '<title>' | awk -F '<title>' '{ print $2 }' | cut -d'<' -f1|sed 's/^[ \t]*//')"
        echo "<li>${title}</li>"
exec 0<&3

To delete trailing whitespace from end of each line, enter:
$ cat input.txt | sed 's/[ \t]*$//' > output.txt
Better remove all leading and trailing whitespace from end of each line:
$ cat input.txt | sed 's/^[ \t]*//;s/[ \t]*$//' > output.txt