Processing the delimited files using cut and awk

From wikipedia, "Delimited data uses specific characters (delimiters) to separate its values. Most database and spreadsheet programs are able to read or save data in a delimited format.".

So how do you process delimited files under Linux shell prompt?

Processing the delimited files using cut

cut command print selected parts of lines from each FILE (or variable) i.e. it remove sections from each line of files:

For example /etc/passwd file is separated using character : delimiters.

To print list of all users, type the following command at shell prompt:

$ cut -d: -f1 /etc/passwd

Output:

root
you
me
vivek
httpd

Where,

• -d : Specifies to use character : as delimiter
• -f1 : Print first field, if you want print second field use -f2 and so on...

Now consider variable service. Let us print out mail word using cut command:
$ service="http mail ssh"
$ echo $service | cut -d' ' -f2

mail

Note that a blank space is used as delimiter.

Processing the delimited files using awk

You can also use awk command for same purpose:
$ awk -F':' '{ print $1 }' /etc/passwd

Output:

root
you
me
vivek
httpd

Where,

• -F: - Use : as fs (delimiter) for the input field separator
• print $1 - Print first field, if you want print second field use $2 and so on

Processing the delimited files using cut or awk is an essential skill for sys admin. Let us say you want to find out if particular service is active or not using shell script (download link):

#!/bin/bash

ports="22 80 25"
service="SSH WEB MAIL"
c=1

echo "Running services status:"

/bin/netstat -tulpn | grep -vE '^Active|Proto' | while read LINE
do
 # get active port name and use : as delimiter
 t=$(echo $LINE | awk '{ print $4}' | cut -d: -f2)
 [ "$t" == "" ] && t=-1 || :
 # get service name from $services and : as delimiter
 sname=$(echo $service | cut -d' ' -f$c)
 sstatus="$sname: No"
 # now compare port
 for i in $ports
 do
  if [ $i -eq $t ]; then
   sstatus="$sname: Ok"
  fi
 done
 # display service status as OK or NO
 echo "$sstatus"
 #next service please
 c=$( expr $c + 1 )
 # break afer 3 services
 [ $c -ge 4 ] && break || :
done

Execute a script:
$ chmod +x script.sh
$ ./script.sh

Output:

SSH: Yes
WEB: No
MAIL: Yes

As you see above script use cut and awk to extract one or many contiguous or non-contiguous fields from shell variables and command output. Download complete working script that sends an email alert to admin user.

Featured Articles:

• 20 Linux System Monitoring Tools Every SysAdmin Should Know
• 20 Linux Server Hardening Security Tips
• 10 Greatest Open Source Software Of 2009
• My 10 UNIX Command Line Mistakes
• Top 5 Email Client For Linux, Mac OS X, and Windows Users
• Top 20 OpenSSH Server Best Security Practices
• Top 10 Open Source Web-Based Project Management Software
• Top 5 Linux Video Editor Software