≡ Menu

HowTo: Read a File Line By Line Using awk

How do I read a file line by line using awk utility under Unix / Linux operating systems?

awk is pattern scanning and text processing language. It is useful for manipulation of data files, text retrieval and processing, and for prototyping and experimenting with algorithms. The name AWK is derived from the surnames of its authors — Alfred Aho, Peter Weinberger, and Brian Kernighan.

By default it process one line at a time. For example following command will simply process one line at a time:
$ cat /etc/passwd | awk '{ print $0}'
Or better try
$ awk '{ print $0}' /etc/passwd

The print command is used to output text. $0 is field name for entire line. By default white space (blank line) act as field separator. You can set new field separator with -F option. For example, to use : as a field separator, enter:
$ awk -F':' '{ print $1 }' /etc/passwd

Above command will print all username using the first field ($1) for current line. You can print username ($1), shell ($7) and login home dir ($6) report as follows:
$ awk -F':' '{ print "User " $1 " login using " $7 " shell with as " $6 " home dir"}' /etc/passwd
Output:

User root login using /bin/bash shell with as /root home dir
User daemon login using /bin/sh shell with as /usr/sbin home dir
User bin login using /bin/sh shell with as /bin home dir
User sys login using /bin/sh shell with as /dev home dir
....
....
User gdm login using /bin/false shell with as /var/lib/gdm home dir
User vivek login using /bin/bash shell with as /home/vivek home dir
User sshd login using /usr/sbin/nologin shell with as /var/run/sshd home dir
Tweet itFacebook itGoogle+ itPDF itFound an error/typo on this page?

{ 13 comments… add one }

  • codeguru February 26, 2009, 6:46 am

    That was pretty cool. Had problems understanding awk. The tip really enlightened me. Thanks

  • TrueColorTech June 1, 2009, 4:08 pm

    Thanks for the tip, made things much easier.

  • sundar June 26, 2009, 2:34 am

    That’s a good tip! By the way, how to read a file, compare a field value then insert record in between using awk? Can you help? Thanks. :-)

  • andre January 19, 2010, 10:33 pm

    Hi great article. One suggestion is that the file names in the awk scripts can be substituted for the standard input.

    e.g. to get the number of lines in a file you could use

    wc -l filename | awk ‘{ print $1 }’

    Andre

  • varpal September 28, 2010, 10:35 am

    Can I compare two column in AWK

  • Ed Dagos September 16, 2011, 1:26 am

    Hi All,
    Is it possible to find a pattern in the file and then print the previous 3 or 4 lines using awk/sed/grep ??
    Regards,
    Ed

  • Trupti October 1, 2011, 5:46 pm

    Exactl, even I wanted to know the same. If anyone could help me with this. :)

    Ed Dagos September 16, 2011

    Hi All,
    Is it possible to find a pattern in the file and then print the previous 3 or 4 lines using awk/sed/grep ??
    Regards,

    • nixCraft October 1, 2011, 8:21 pm

      Search for “foo” in data.txt and display 3 lines of leading context before matching line:

      grep -B3 "foo" data.txt

      The -A options display $NUM lines of trailing context after matching lines:

      grep -A3 "foo" data.txt
      • Ed Dagos October 3, 2011, 8:48 pm

        Thanks Vivek for your response. However, on Sun OS 5.10 Options -B and -A do not work. Would you mind to tell me what environment have you tested it.
        Regards,
        Ed

  • owahid November 4, 2011, 12:59 am

    Hi,
    If I have 8 lines (8 row with 1 column) in a file, can I print every 2 row in each row? Like
    1
    2
    3
    4
    .
    .
    wil be
    12
    34
    ..
    Thanks

  • muugii March 31, 2012, 3:57 am

    help me please
    cat $file | awk ‘{ split($1, a, “;”) ;
    if( a[2]= “wlan1″ ) # how to line print
    if( a[4]!=0.00) }’

  • Robert Edward Johnson June 10, 2012, 3:19 am

    Is there any way to use awk (or any other aspect of unix shell scripting) to “declare” fields from position a to position b etc in the outline of the input, and reference those fields, IF there are no “automatic separators like a spacee, semicolon, etc. I have input with no separators and would love to use something “like” the usual while read LINE ; do processinputintooutput done < filename, but I'm not sure it can be done in shell scripting without separators, though it sounds like awk would work if I DID have separators.

  • Vikram December 15, 2014, 5:39 pm

    i have a file a.dat with following input

    123456789

    the first 5 digits represents ABC
    the next 4 digits represents XYZ

    how do i use awk to do the following

    If XYZ = 6789 then print ABCXYZ in a new file
    else
    skip that line

Leave a Comment