HowTo: Read a File Line By Line Using awk

How do I read a file line by line using awk utility under Unix / Linux operating systems?

awk is pattern scanning and text processing language. It is useful for manipulation of data files, text retrieval and processing, and for prototyping and experimenting with algorithms. The name AWK is derived from the surnames of its authors — Alfred Aho, Peter Weinberger, and Brian Kernighan.

ADVERTISEMENTS

By default it process one line at a time. For example following command will simply process one line at a time:
$ cat /etc/passwd | awk '{ print $0}'
Or better try
$ awk '{ print $0}' /etc/passwd

The print command is used to output text. $0 is field name for entire line. By default white space (blank line) act as field separator. You can set new field separator with -F option. For example, to use : as a field separator, enter:
$ awk -F':' '{ print $1 }' /etc/passwd

Above command will print all username using the first field ($1) for current line. You can print username ($1), shell ($7) and login home dir ($6) report as follows:
$ awk -F':' '{ print "User " $1 " login using " $7 " shell with as " $6 " home dir"}' /etc/passwd
Output:

User root login using /bin/bash shell with as /root home dir
User daemon login using /bin/sh shell with as /usr/sbin home dir
User bin login using /bin/sh shell with as /bin home dir
User sys login using /bin/sh shell with as /dev home dir
....
....
User gdm login using /bin/false shell with as /var/lib/gdm home dir
User vivek login using /bin/bash shell with as /home/vivek home dir
User sshd login using /usr/sbin/nologin shell with as /var/run/sshd home dir
🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:
CategoryList of Unix and Linux commands
File Managementcat
FirewallCentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network Utilitiesdig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNCentOS 8 Debian 10 Firewall Ubuntu 20.04

ADVERTISEMENTS
14 comments… add one
  • codeguru Feb 26, 2009 @ 6:46

    That was pretty cool. Had problems understanding awk. The tip really enlightened me. Thanks

  • TrueColorTech Jun 1, 2009 @ 16:08

    Thanks for the tip, made things much easier.

  • sundar Jun 26, 2009 @ 2:34

    That’s a good tip! By the way, how to read a file, compare a field value then insert record in between using awk? Can you help? Thanks. :-)

  • andre Jan 19, 2010 @ 22:33

    Hi great article. One suggestion is that the file names in the awk scripts can be substituted for the standard input.

    e.g. to get the number of lines in a file you could use

    wc -l filename | awk '{ print $1 }'

    Andre

  • varpal Sep 28, 2010 @ 10:35

    Can I compare two column in AWK

  • Ed Dagos Sep 16, 2011 @ 1:26

    Hi All,
    Is it possible to find a pattern in the file and then print the previous 3 or 4 lines using awk/sed/grep ??
    Regards,
    Ed

  • Trupti Oct 1, 2011 @ 17:46

    Exactl, even I wanted to know the same. If anyone could help me with this. :)

    Ed Dagos September 16, 2011

    Hi All,
    Is it possible to find a pattern in the file and then print the previous 3 or 4 lines using awk/sed/grep ??
    Regards,

    • 🐧 nixCraft Oct 1, 2011 @ 20:21

      Search for “foo” in data.txt and display 3 lines of leading context before matching line:

      grep -B3 "foo" data.txt

      The -A options display $NUM lines of trailing context after matching lines:

      grep -A3 "foo" data.txt
      • Ed Dagos Oct 3, 2011 @ 20:48

        Thanks Vivek for your response. However, on Sun OS 5.10 Options -B and -A do not work. Would you mind to tell me what environment have you tested it.
        Regards,
        Ed

  • owahid Nov 4, 2011 @ 0:59

    Hi,
    If I have 8 lines (8 row with 1 column) in a file, can I print every 2 row in each row? Like
    1
    2
    3
    4
    .
    .
    wil be
    12
    34
    ..
    Thanks

  • muugii Mar 31, 2012 @ 3:57

    help me please
    cat $file | awk ‘{ split($1, a, “;”) ;
    if( a[2]= “wlan1” ) # how to line print
    if( a[4]!=0.00) }’

  • Robert Edward Johnson Jun 10, 2012 @ 3:19

    Is there any way to use awk (or any other aspect of unix shell scripting) to “declare” fields from position a to position b etc in the outline of the input, and reference those fields, IF there are no “automatic separators like a spacee, semicolon, etc. I have input with no separators and would love to use something “like” the usual while read LINE ; do processinputintooutput done < filename, but I'm not sure it can be done in shell scripting without separators, though it sounds like awk would work if I DID have separators.

  • Vikram Dec 15, 2014 @ 17:39

    i have a file a.dat with following input

    123456789

    the first 5 digits represents ABC
    the next 4 digits represents XYZ

    how do i use awk to do the following

    If XYZ = 6789 then print ABCXYZ in a new file
    else
    skip that line

  • AS Jan 7, 2016 @ 20:41

    % awk 'match($0, /.*([0-9][0-9][0-9][0-9])/, arr) {if (arr[1] == 6789) {print $0}}' a.dat > out_file

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre>, <code>...</code> and <kbd>...</kbd> for code samples.