Bash Read Comma Separated CSV File on Linux / Unix

last updated in Categories , , , , , , , , ,

How do I read comma separated CVS file under UNIX / Linux / BSD / MacOS bash script? My sample file is as follows
FirstName LastName,DOB,SSN,Telephone,Status
How can I parse a CSV file in Bash?

A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. A CSV file stores tabular data in plain text format. Each line of the file is a data record. You can use while shell loop to read comma-separated cvs file. IFS variable will set cvs separated to , (comma). The read command will read each line and store data into each field. Let us see how to parse a CSV file in Bash running under Linux, macOS, *BSD or Unix-like operating systems.

ADVERTISEMENTS


Bash Read Comma Separated CSV File

The syntax is as follows phrase a CSV file named input.csv:

while IFS=, read -r field1 field2
do
    echo "$field and $field2"
done < input.csv

How to parse a CSV file in Bash

Create a file called test.sh using a text editor such as vim command/nano command:
vim test.sh
Append the following code:

#!/bin/bash
# Purpose: Read Comma Separated CSV File
# Author: Vivek Gite under GPL v2.0+
# ------------------------------------------
INPUT=data.cvs
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read flname dob ssn tel status
do
	echo "Name : $flname"
	echo "DOB : $dob"
	echo "SSN : $ssn"
	echo "Telephone : $tel"
	echo "Status : $status"
done < $INPUT
IFS=$OLDIFS

Run the test.sh file shell script as follows by setting up a execute permissions:
chmod +x test.sh
./test.sh
bash test.sh
sh test.sh

Linux and Unix Bash Read Comma Separated CSV File
Parse a CSV file in Bash

CSV application support

The CSV file format is supported by spreadsheets and database management systems, including LibreOffice Calc, and Apache OpenOffice Calc.

comma-separated values (CSV) file in openoffice
Importing and Exporting CSV Files

How to open/import a text CSV file in calc

One can read comma separated CSV file using GUI app too.

  1. Start calc
  2. Choose File > Open.
  3. Locate the CSV file that you want to open.
  4. If the file has a *.csv extension, select the file.
  5. Click Open.
  6. The Text Import dialog opens.
  7. Specify the options to divide the text in the file into columns.
  8. Click OK.

Many Linux and Unix command line utility programs such as cut, paste, join, sort, uniq, awk, sed can split files on a comma delimiter, and can therefore process simple CSV files. For example:
awk -F',' '{ print $1 " " $2 }'

Conclusion

You learned how to read and parse comma-separated (CSV) file under a Linux or Unix-like system using bash while loop and read command.

ADVERTISEMENTS

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Notable Replies

  1. The awk program is a great tool for parsing CSV files. You can use it within a bash script.
    For example, this line would print the first and second fields in a CSV file:

    awk -F, '{ print $1,$2 }' filename.csv

Continue the discussion www.nixcraft.com

1 more reply

Participants

Historical Comment Archive

16 comment

  1. script that worked for me

    #!/bin/bash
INPUT=./Zabbix_template_item.csv
OLDIFS=$IFS
IFS=";"
[ ! -f $INPUT ] &while read Host Group itItemType itKey itValueType itDescription itDelay itHistory itTrends itDataType itUnits itMultiplier itDelta itFormula itParams itSnmpCom itSnmpOid itApplication
do
        echo "Host : $Host"
        echo "Group : $Group"
        echo "itItemType : $itItemType"
        echo "itKey : $itKey"
        echo "itValueType : $itValueType"
        echo "itDescription : $itDescription"
        echo "itDelay : $itDelay"
        echo "itHistory : $itHistory"
        echo "itTrends : $itTrends"
        echo "itDataType : $itDataType"
        echo "itUnits : $itUnits"
        echo "itMultiplier : $itMultiplier"
        echo "itDelta : $itDelta"
        echo "itFormula : $itFormula"
        echo "itParams : $itParams"
        echo "itSnmpCom : $itSnmpCom"
        echo "itSnmpOid : $itSnmpOid"
        echo "itApplication : $itApplication"

done < $INPUT
IFS=$OLDIFS

    2. i have used the same code to read my csv file but i cant read the last row of my csv file using while loop. please help.

          1. you can get the last line when you access the variables again outside the loop. using the example discussed in the post:

            —————————–
            while read flname dob ssn tel status
            …..
            done < $INPUT
            echo $flname # <—– this will be the last line of the CSV

  2. I have two Csv files one is for daily report purposed and another is for daily data purposed.
    masterreport.csv file format is

    CODE,1-May-12,2-May-12,3-May-12,4-May-12,5-May-12,
    205,0,0,0,0,0,
    23070,0,0,0,0,0,
    Interface error response CRBT fail,0,1,0,0,0,
    C2S exception,0,1,2,0,2,
    Interface Customer Recharge Not Allowed For Receiver,2240,2078,2050,2007,2363,

    —–Many more up to 45 Rows and 32 column are there
    and every day we have to update the corresponding date part from below mention file.

    dailyreport.csv
    C2S ERROR EXCEPTION TAKING TIME TILL VALIDATION , 624
    CHNL_ERROR_SNDR_AMT_NOTBETWEEN_MINMAX , 56
    Card Group Slab Suspended , 7096

    like how many error’s are coming on that day we have to update the main report , which are not match daily report we put the value 0 on main report.

    i didn’t found any logic how to do this , can any body help this

  3. 15 years working with csv files in bash and I didn’t know this method!

    Thanks a lot.

  7. This was exactly what I needed! I’m using it to import account data from a Zimbra server backup.

    IE account names are stored in the CSV and the script runs the import command with the $flname variable in the appropriate spots.

    I do have a question, How does it know to hit the next line and not just read the first line every time? My assumption is that is what the $IFS & $OLDIFS variables do. But I’m not sure.

  8. This method is only for regular simplest version of CSV. very often fields are in quotation marks and it contains comma. In that situation for row
    content1,”content,number2″,content3
    read c1 c2 c3 assign
    c1=’content1′
    c2='”content”
    c3=’number2″

    then this method is not as universal as it should be.

    last problem, very often last row in csv file is not ended with new line. in that situation read has some problem with fetching last row.

    summarizing. this is the simplest way for reading the simplest cvs formatting.

    Still, have a question? Get help on our forum!

Tagged as: Tags , , , , ,