How do I read a file field-by-field under UNIX / Linux / BSD Bash shell? My sample input data file is as follows:
device1,deviceType,major,minor,permissions
device2,deviceType,major,minor,permissions
...
....
.
deviceN,deviceTypeN,major,minor,permissions
For each line I need to construct and execute a shell command as follows:
/path/to/deviceMaker --context=$1 -m $permissions $device2 $deviceType $major $minor
You can use a while loop along with the read command, Internal Field Separator (IFS), and HERE STRINGS as follows:
#!/bin/bash input=/path/to/data.txt [ $# -eq 0 ] && { echo "Usage: $0 arg1"; exit 1; } arg="$1" cmd=/path/to/deviceMaker while read -r line do IFS=, read -r f1 f2 f3 f4 f5 <<<"$line" # quote fields if needed $cmd --context="$arg" -m $f5 $f1 $f2 $f3 $f4 done <"$input"
See also
You should follow me on twitter here or grab rss feed to keep track of new changes.
Featured Articles:
- 30 Handy Bash Shell Aliases For Linux / Unix / Mac OS X
- Top 30 Nmap Command Examples For Sys/Network Admins
- 25 PHP Security Best Practices For Sys Admins
- 20 Linux System Monitoring Tools Every SysAdmin Should Know
- 20 Linux Server Hardening Security Tips
- Linux: 20 Iptables Examples For New SysAdmins
- Top 20 OpenSSH Server Best Security Practices
- Top 20 Nginx WebServer Best Security Practices
- 20 Examples: Make Sure Unix / Linux Configuration Files Are Free From Syntax Errors
- 15 Greatest Open Source Terminal Applications Of 2012
- My 10 UNIX Command Line Mistakes
- Top 10 Open Source Web-Based Project Management Software
- Top 5 Email Client For Linux, Mac OS X, and Windows Users
- The Novice Guide To Buying A Linux Laptop
{ 5 comments… read them below or add one }
"$cmd --context="$arg" -m $f5 $f1 $f2 $f3 $f4"That line will fail if any of the arguments contain whitespace. They should be quoted:
$cmd --context="$arg" -m "$f5" "$f1" "$f2" "$f3" "$f4"—————
The example can be coded more efficiently:
> The example can be coded more efficiently:
Agreed, but consider using AWK instead if you get performance problems.
@Vinod
Do you have any evidence that a [read/while loop] script would be less efficient than a awk script ?
If the file is longer than X lines, an awk script will be faster because it doesn’t have to interpret a loop for every line.
X may be anywhere from a dozen to a gross, or perhaps more or less.
> Do you have any evidence that a [read/while loop] script would be less efficient than a awk script ?
This is based upon my own experience. awk always performed well when we skipped the ksh [read/while] for file size > 1G. We used time command to get exact values. See thread at nixcraft forum about awk vs shell for counting ips. Also, code in C will not help or speed up (or not worth your time) if awk can not perform well. awk is a real winner when it comes to processing large file size. I have tested and worked on real UNIX (HP-UX v11.0) not on GNU/Linux (no offense, Linux is wonderful too but OIL industry likes to spends its money on HP stuff) tools so YMMV.