Ksh Read a File Line By Line ( UNIX Scripting )

Author: Vivek Gite Last updated: August 23, 2015 18 comments

How do I read a file line by line using KSH shell scripting under UNIX like operating systems?

You can use the while loop and read command to read a text file line by line under KSH.

KSH read while loop syntax

#!/bin/ksh
file="/path/to/file.txt"
# while loop
while IFS= read -r line
do
        # display line or do somthing on $line
	echo "$line"
done <"$file"

In this example, you are reading file separated by | fields. Sample domains.txt:

cyberciti.biz|74.86.48.99
nixcraft.com|75.126.168.152
theos.in|75.126.168.153
cricketnow.in|75.126.168.154
vivekgite.com|75.126.168.155

#!/bin/ksh
# set the Internal Field Separator to a pipe symbol
IFS='|'
 
# file name
file=/tmp/domains.txt
 
# use while loop to read domain and ip 
while read domain ip
do
	print "$domain has address $ip"
done <"$file"

However, following is recommend syntax to set the Internal field separator (see discussion below):

#!/bin/ksh
# file name
file=/tmp/domains.txt
 
# use while loop to read domain and ip 
# set the Internal Field Separator to a pipe symbol
while IFS=\| read domain ip
do
	print "$domain has address $ip"
done <"$file"

Suggested readings:

ksh man page

Updated for accuracy!

🐧 If you liked this page, please support my work on Patreon or with a donation.
🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:

RSS feed or Weekly email newsletter
Share on Twitter • Facebook • 18 comments... add one ↓

UP NEXT

Category	List of Unix and Linux commands
File Management	cat
Firewall	Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04
Network Utilities	dig • host • ip • nmap
OpenVPN	CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04
Package Manager	apk • apt
Processes Management	bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time
Searching	grep • whereis • which
User Information	groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w
WireGuard VPN	Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04

18 comments… add one

Daniel J. Doughty Nov 20, 2009 @ 17:00

Thanks, I usually don’t like doing my heavy lifting with shell scripting. Still, this is useful information because sometimes the shell is all you have in the environment you’re working in.

Dan

Reply Link
Joel Nov 20, 2009 @ 22:15

Thanks for the scripting tips, they really are the key to effective system administration tasks. The differences between the shells is new to me, I usually just use bash since it’s the default on the systems I use. I looked up the Korn shell on Wikipedia and it reports that it has backwards compatibility to the Bourne shell (which is Bash’s predecessor). On my system I did an experiment to see if the scripts would work with Bash and changed the #!/bin/ksh to #!/bin/sh. The first one works find, the second doesn’t – looks like it doesn’t support the “IFS” environment label. Interesting.

Now, if someone can explain the different between this alphabet soup and sym links that would be helpful (Ubuntu 8.04.1). Never heard of ‘dash’!

-rwxr-xr-x 1 root root 702160 May 12 2008 /bin/bash
-rwxr-xr-x 1 root root 79988 Mar 12 2008 /bin/dash
lrwxrwxrwx 1 root root 21 Nov 20 14:05 /bin/ksh -> /etc/alternatives/ksh
lrwxrwxrwx 1 root root 4 Apr 24 2008 /bin/sh -> dash

Reply Link
Shantanu Oak Nov 21, 2009 @ 5:37

When there are a few lines to be read, I do always use the following method.

#!/bin/ksh
while read line
do
echo “$line”
done << iplist
cyberciti.biz|74.86.48.99
nixcraft.com|75.126.168.152
iplist

Reply Link
🐧 nixCraft Nov 21, 2009 @ 9:44
@Joel,

The original sh is not available under Linux. dash is the standard command interpreter for the system. The current version of dash is in the process of being changed to conform with the POSIX 1003.2 and 1003.2a specifications for the shell. This version has many features which make it appear similar in some respects to the Korn shell, but it is not a Korn shell clone.

/etc/alternatives/ksh should point to to ksh93. Run the following
```
ls -l /etc/alternatives/ksh
```
sym links allows to use different version of ksh without breaking anything. update-alternatives command maintain symbolic links determining default commands for various system commands, and java. See man page
```
man ksh
man dash
man update-alternatives
```
Reply Link
Philippe Petrinko Nov 21, 2009 @ 12:33

dash is standard shell under UBUNTU,
(and should have been under Debian Lenny)
http://en.wikipedia.org/wiki/Debian_Almquist_shell

Reply Link
Brock Noland Nov 24, 2009 @ 22:27

Thanks for making people aware of native shell string splitting, but I would reset IFS.

Reply Link
Chris F.A. Johnson Nov 30, 2009 @ 14:41

The safe way to change IFS in a ‘while read’ loop is:
while IFS=\| read a b c do : whatever done

Then it doesn’t affect anything but the read command.

Reply Link
Philippe Petrinko Nov 30, 2009 @ 17:19

To Chris : short and simple – How nice!
Should have think of this good old shell syntax
Good example of KISS principle … (ask Wikipedia)

Reply Link
- Nicolas Mar 17, 2017 @ 12:54
  
  You are speaking with C F.A. Johnson…
  (Ask Google who is he)
  
  Reply Link

ishan Feb 24, 2012 @ 6:49

while reading line by line like this if you want to take user desire input
eg :-
cat use |while read mo
do
read -r o
if [[ $o = ‘y’ ]]
then
print “$mo”
else
print “kill me”
fi
done

In this case o/p will be always kill me …….user input wouldn’t be there so …..how to take user input in that

Reply Link

Philippe Petrinko Feb 24, 2012 @ 11:35

@ishan

Solution is to separate user input from data input.
Use Vivek’s tutorial:
http://bash.cyberciti.biz/guide/Main_Page

A solution may be:

#!/bin/bash
# File descriptor #0 is stdin
# File descriptor #1 is stdout
# File descriptor #2 is stderr
# Let us assign extra file descriptor to file for input

# open for input file named "data" with fd #3 
exec 3< data

# loop on fd #3
while read <&3
do
	# ask user input, by default on stdin
	read -p "Print next line of data ? (y/n):" answer 

	# to debug, uncomment the following line
	# echo "Answer:${answer}:"

	# branch on answer, casted to lowercase
	if [[ "${answer,,}" = 'y' ]]
	then
		# user wants to see data 
		# which is by default read in builtin REPLY variable
		echo $REPLY
	else
		# user wants to quit 
		# let us terminate while loop
		break
	fi
done 
 
# Close fd #3
exec 3<&-

HTH, Try it and let us know your solution.

— Philippe

Reply Link

Prashanth Feb 14, 2014 @ 11:22

Dear Experts,
Greetings..!
I have a scenario where in
we need read file (.csv) row by row and
(
Select the value for the column “Q2014”.
which further segregated into (Q1 2014 ,Q2 2014,Q3 2014,Q4 2014) we need to get the Sum Q1 2014 for all products.
If the Sum(Q1 2014 ) > sum(Q2 2014) if the difference is > 5% then send a mail.
)
Your help in highly appreciable.

Thanks in Advance,

Reply Link

Mike S Mar 12, 2014 @ 16:14

Prashanth,
This is probably better handled by an application than a Korn shell script, but it sounds like an interesting challenge, so here goes.
First, it isn’t clear to me from your question how your .csv file is formatted, but I’m going to assume the following:

Q1 2013,Q2 2013,Q3 2013,Q4 2013,Q1 2014,Q2 2014
5000,6000,7000,8000,9000,10000
500,600,700,800,900,1000
(etc.)

If your CSV file has a different format, you should be able to modify the solution below to match it.

#!/bin/ksh
let linenum=1
IFS=','
let col1=-1
let col2=-1
let sum1=0
let sum2=0
while read fline
do
  set -A farray $fline
  let numcols=${#farray[*]}
  if [[ $linenum -eq 1 ]]; then
    for (( x=1 ; $x <= $numcols ; x+=1 ))
    do
      if [[ $farray[x] == "Q1 2014" ]]; then
        let col1=$x
      fi
      if [[ $farray[x] == "Q2 2014" ]]; then
        let col2=$x
      fi
    done
    if [[ $col1 -eq -1 || $col2 -eq -1 ]]; then
      echo 'One or both column headers not found'
      exit 1
    fi
  else
    let sum1+=${farray[$col1]}
    let sum2+=${farray[$col2]}
  fi
  let linenum+=1
done 
Obviously, the echo statement would be replaced by whatever you use to send an email from the script. I've also hardcoded the filename and column header strings for simplicity, but I assume you'd make those values script inputs.

This works for me, but I might have missed something (I'm not a scripting expert by any means) so comments/corrections from experts are welcomed.
Edited by mods

Reply Link

Mike S Mar 12, 2014 @ 16:19
Oops – mis-copied part of the script!
```
done  sum1
```
Should have been
```
done < "datafile.csv"
```
Reply Link

vignesh Apr 14, 2014 @ 2:47

could any one please tell me how to declare sql statement inside ‘do’ and get output in a text

thanks

Reply Link
Neelam Dec 11, 2014 @ 10:32

Dear Experts,
Greetings..!
I have a scenario where in
we need read file (.csv) row by row and
(
And print the column header with its value
i.e input.csv
Q1,Q2,Q3
n1 , n2, n3
then we want to print this in text file as
output.csv
Q1 n1^]Q2 n2^]Q3 n3

)
Your help in highly appreciable.

Thanks in Advance,

Reply Link
Sreenivasulu May 13, 2015 @ 12:24

Hi Guys,

I’m having a scenario.
1. UNIX script needs to pick a sample.schema file and a source.txt file(fixed length file i.e no delimeter)
2. Based on the schema file it should split the source file data into columns in a csv format file.
3. Finally source file data should be viewed in 3 columns with respect to schema file.

sample.schema (record length =54):-

record
{final_delim=none, delim=none, quote=none}
(
WS_PH_PLAN_NUM:string[16];
WS_PH_PART_ID:string[8];
WS_PH_PART_NAME:string[30];
)

Source.txt (Record length of each record is 54) :
RG0013001056704RH1001 BLADENSBURG, KEETON A
RG0013001431587RH1001 HOSTETTER, ASHLYN M
RG0013001484327RH1001 BRETHREN, KENNY K

Reply Link
Hari Dec 15, 2015 @ 8:47
I am trying to get filesystem /mysqlshare in all my servers ie i have 300 servers trying to get details of all.But for now trying get only 2 server details from txt file ie(text.txt) which have
server1_name
server2_name
when i try this script its only reading 1st server details and getting exit from script
```
#!/bin/ksh
file="/home/mysqladm/server_list/server_list.txt"
# while loop
while IFS= read -r a
do
        # display line or do somthing on $line
        output=`ssh $a df -h | grep mysqlshare`
#echo $a,$output
echo $a ,$output >>/home/mysqladm/server_list/output.txt

done <"$file"
```
Can someone help me out..
Reply Link