Unix / Linux Shell: Parse Text CVS File Separator By Field

example.com,username,groupname,homedir,md5password,permission,secondarygroup

I need to extract each of these example.com,username,groupname,homedir,md5password,permission,secondarygroup fields and passed to the different system utilities. How do I write a shell script to automate this task and use the bash shell to parse a text file?

[donotprint]

1. while loop command – The while statement is used to execute a list of commands repeatedly.
2. read command – Use the read command if you want to receive input while running a script. The read statement accepts input from the keyboard or file.
3. $IFS – The Internal Field Separator (IFS) that is used for word splitting after expansion and to split lines into words with the read builtin command.

Examples

In this example, read a Comma-separated values (CVS) text file using combination of while and read commands. Here is a sample file.cvs:

cyberciti.biz,foo,bar,user1,password1,homedir1,chroot
nixcraft.com,oof,rab,user2,password2,homedir,nochroot

Create a shell script as follows:

#!/bin/bash
input="/path/to/your/input/file.cvs"
# Set "," as the field separator using $IFS 
# and read line by line using while read combo 
while IFS=',' read -r f1 f2 f3 f4 f5 f6 f7
do 
  echo "$f1 $f2 $f3 $f4 $f5 $f6 $f7"
done < "$input"

Save and close the file. Run it as follows:
$ ./script.sh
Sample outputs:

cyberciti.biz foo bar user1 password1 homedir1 chroot
nixcraft.com oof rab user2 password2 homedir nochroot

You can pass f1, f2 and other fields as per your requirements to other Linux or Unix commands:

#!/bin/bash
input="/path/to/your/input/file.cvs"
while IFS=',' read -r f1 f2 f3 f4 f5 f6 f7
do 
  /path/to/useradd -d "$f4" "$f2"
done < "$input"

The IFS is a special shell variable. The Internal Field Separator (IFS) that is used for word splitting after expansion and to split lines into words with the read builtin command.

Parse a CVS file using awk

Awk use the field separator called FS and it does not use the name IFS that is used by the POSIX-compliant shells as described above. The syntax is as follows:

awk 'BEGIN { FS = "," } ; { do_something_here }' < input.cvs
awk 'BEGIN { FS = "," } ; { do_something_here } END{ clean_up_here }' < input.cvs
awk 'BEGIN { FS = "," } ; { print }' < input.cvs

In this example, display domain name using from file.cvs:

awk 'BEGIN { FS = "," } ; { print $1 }' < file.svs

Sample outputs:

cyberciti.biz
nixcraft.com

In this final example, pass f1, f2 and other fields as per your requirements to other Linux or Unix commands using awk’s system(cmd) to executes cmd:

awk 'BEGIN { FS = "," } ; { cmd="/path/to/useradd -d" $4 " " $f2; system(cmd) }' < file.svs

See also

• Ksh Read a File Line By Line ( UNIX Scripting )
• Man pages – awk(1)