How do I read a file line by line using awk utility under Unix / Linux operating systems?
awk is pattern scanning and text processing language. It is useful for manipulation of data files, text retrieval and processing, and for prototyping and experimenting with algorithms. The name AWK is derived from the surnames of its authors — Alfred Aho, Peter Weinberger, and Brian Kernighan.
By default it process one line at a time. For example following command will simply process one line at a time:
$ cat /etc/passwd | awk '{ print $0}'
Or better try
$ awk '{ print $0}' /etc/passwd
The print command is used to output text. $0 is field name for entire line. By default white space (blank line) act as field separator. You can set new field separator with -F option. For example, to use : as a field separator, enter:
$ awk -F':' '{ print $1 }' /etc/passwd
Above command will print all username using the first field ($1) for current line. You can print username ($1), shell ($7) and login home dir ($6) report as follows:
$ awk -F':' '{ print "User " $1 " login using " $7 " shell with as " $6 " home dir"}' /etc/passwd
Output:
User root login using /bin/bash shell with as /root home dir User daemon login using /bin/sh shell with as /usr/sbin home dir User bin login using /bin/sh shell with as /bin home dir User sys login using /bin/sh shell with as /dev home dir .... .... User gdm login using /bin/false shell with as /var/lib/gdm home dir User vivek login using /bin/bash shell with as /home/vivek home dir User sshd login using /usr/sbin/nologin shell with as /var/run/sshd home dir
14 comment
That was pretty cool. Had problems understanding awk. The tip really enlightened me. Thanks
Thanks for the tip, made things much easier.
That’s a good tip! By the way, how to read a file, compare a field value then insert record in between using awk? Can you help? Thanks. :-)
Hi great article. One suggestion is that the file names in the awk scripts can be substituted for the standard input.
e.g. to get the number of lines in a file you could use
wc -l filename | awk '{ print $1 }'Andre
Can I compare two column in AWK
Hi All,
Is it possible to find a pattern in the file and then print the previous 3 or 4 lines using awk/sed/grep ??
Regards,
Ed
Exactl, even I wanted to know the same. If anyone could help me with this. :)
Ed Dagos September 16, 2011
Hi All,
Is it possible to find a pattern in the file and then print the previous 3 or 4 lines using awk/sed/grep ??
Regards,
Search for “foo” in data.txt and display 3 lines of leading context before matching line:
The -A options display $NUM lines of trailing context after matching lines:
Thanks Vivek for your response. However, on Sun OS 5.10 Options -B and -A do not work. Would you mind to tell me what environment have you tested it.
Regards,
Ed
Hi,
If I have 8 lines (8 row with 1 column) in a file, can I print every 2 row in each row? Like
1
2
3
4
.
.
wil be
12
34
..
Thanks
help me please
cat $file | awk ‘{ split($1, a, “;”) ;
if( a[2]= “wlan1” ) # how to line print
if( a[4]!=0.00) }’
Is there any way to use awk (or any other aspect of unix shell scripting) to “declare” fields from position a to position b etc in the outline of the input, and reference those fields, IF there are no “automatic separators like a spacee, semicolon, etc. I have input with no separators and would love to use something “like” the usual while read LINE ; do processinputintooutput done < filename, but I'm not sure it can be done in shell scripting without separators, though it sounds like awk would work if I DID have separators.
i have a file a.dat with following input
123456789
the first 5 digits represents ABC
the next 4 digits represents XYZ
how do i use awk to do the following
If XYZ = 6789 then print ABCXYZ in a new file
else
skip that line
% awk 'match($0, /.*([0-9][0-9][0-9][0-9])/, arr) {if (arr[1] == 6789) {print $0}}' a.dat > out_file