Bash For Loop Examples

by on October 31, 2008 · 231 comments· LAST UPDATED May 30, 2013

in , ,

How do I use bash for loop to repeat certain task under Linux / UNIX operating system? How do I set infinite loops using for statement? How do I use three-parameter for loop control expression?

A 'for loop' is a bash programming language statement which allows code to be repeatedly executed. A for loop is classified as an iteration statement i.e. it is the repetition of a process within a bash script.
Tutorial details
DifficultyEasy (rss)
Root privilegesNo
RequirementsLinux/Unix/*BSD/OS X
GNU bash shell
Estimated completion time10 minutes

For example, you can run UNIX command or task 5 times or read and process list of files using a for loop. A for loop can be used at a shell prompt or within a shell script itself.

for loop syntax

Numeric ranges for syntax is as follows:

for VARIABLE in 1 2 3 4 5 .. N
do
	command1
	command2
	commandN
done

OR

for VARIABLE in file1 file2 file3
do
	command1 on $VARIABLE
	command2
	commandN
done

OR

for OUTPUT in $(Linux-Or-Unix-Command-Here)
do
	command1 on $OUTPUT
	command2 on $OUTPUT
	commandN
done

Examples

This type of for loop is characterized by counting. The range is specified by a beginning (#1) and ending number (#5). The for loop executes a sequence of commands for each member in a list of items. A representative example in BASH is as follows to display welcome message 5 times with for loop:

#!/bin/bash
for i in 1 2 3 4 5
do
   echo "Welcome $i times"
done

Sometimes you may need to set a step value (allowing one to count by two's or to count backwards for instance). Latest bash version 3.0+ has inbuilt support for setting up ranges:

#!/bin/bash
for i in {1..5}
do
   echo "Welcome $i times"
done

Bash v4.0+ has inbuilt support for setting up a step value using {START..END..INCREMENT} syntax:

#!/bin/bash
echo "Bash version ${BASH_VERSION}..."
for i in {0..10..2}
  do
     echo "Welcome $i times"
 done

Sample outputs:

Bash version 4.0.33(0)-release...
Welcome 0 times
Welcome 2 times
Welcome 4 times
Welcome 6 times
Welcome 8 times
Welcome 10 times

The seq command (outdated)

WARNING! The seq command print a sequence of numbers and it is here due to historical reasons. The following examples is only recommend for older bash version. All users (bash v3.x+) are recommended to use the above syntax.

The seq command can be used as follows. A representative example in seq is as follows:

#!/bin/bash
for i in $(seq 1 2 20)
do
   echo "Welcome $i times"
done

There is no good reason to use an external command such as seq to count and increment numbers in the for loop, hence it is recommend that you avoid using seq. The builtin command are fast.

Three-expression bash for loops syntax

This type of for loop share a common heritage with the C programming language. It is characterized by a three-parameter loop control expression; consisting of an initializer (EXP1), a loop-test or condition (EXP2), and a counting expression (EXP3).

for (( EXP1; EXP2; EXP3 ))
do
	command1
	command2
	command3
done

A representative three-expression example in bash as follows:

#!/bin/bash
for (( c=1; c<=5; c++ ))
do
   echo "Welcome $c times"
done
 

Sample output:

Welcome 1 times
Welcome 2 times
Welcome 3 times
Welcome 4 times
Welcome 5 times

How do I use for as infinite loops?

Infinite for loop can be created with empty expressions, such as:

#!/bin/bash
for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
done

Conditional exit with break

You can do early exit with break statement inside the for loop. You can exit from within a FOR, WHILE or UNTIL loop using break. General break statement inside the for loop:

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (disaster-condition)
  then
	break       	   #Abandon the loop.
  fi
  statements3          #While good and, no disaster-condition.
done

Following shell script will go though all files stored in /etc directory. The for loop will be abandon when /etc/resolv.conf file found.

#!/bin/bash
for file in /etc/*
do
	if [ "${file}" == "/etc/resolv.conf" ]
	then
		countNameservers=$(grep -c nameserver /etc/resolv.conf)
		echo "Total  ${countNameservers} nameservers defined in ${file}"
		break
	fi
done

Early continuation with continue statement

To resume the next iteration of the enclosing FOR, WHILE or UNTIL loop use continue statement.

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (condition)
  then
	continue   #Go to next iteration of I in the loop and skip statements3
  fi
  statements3
done

This script make backup of all file names specified on command line. If .bak file exists, it will skip the cp command.

#!/bin/bash
FILES="$@"
for f in $FILES
do
        # if .bak backup file exists, read next file
	if [ -f ${f}.bak ]
	then
		echo "Skiping $f file..."
		continue  # read next file and skip cp command
	fi
        # we are hear means no backup file exists, just use cp command to copy file
	/bin/cp $f $f.bak
done

Check out related media

This tutorial is also available in a quick video format. The video shows some additional and practical examples such as converting all flac music files to mp3 format, all avi files to mp4 video format, unzipping multiple zip files or tar balls, gathering uptime information from multiple Linux/Unix servers, detecting remote web-server using domain names and much more.



Video 01: 15 Bash For Loop Examples for Linux / Unix / OS X Shell Scripting

Recommended readings:

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 231 comments… read them below or add one }

1 Jadu Saikia November 2, 2008 at 3:37 pm

Nice one. All the examples are explained well, thanks Vivek.

seq 1 2 20
output can also be produced using jot

jot – 1 20 2

The infinite loops as everyone knows have the following alternatives.

while(true)
or
while :

//Jadu

Reply

2 Sean November 4, 2008 at 2:20 am

The last example can also be produced without the ” in $FILES”:

#!/bin/sh

for f
do

# For-Loop body

done

If the ” in …” is excluded, the loop will run as if “in $@” was given.

Reply

3 Andreas November 13, 2008 at 4:53 am

Nice explanation tutorial.

Reply

4 Manish November 25, 2008 at 6:33 am

hey vivek i tried the following syntax for for loop suggested by u but both dint work…
1.
#!/bin/bash
for (( c=1; c<=5; c++ ))
do
echo “Welcome $c times…”
done

2.
#!/bin/bash
for i in {1..5}
do
echo “Welcome $i times”
done

got error for both the syntax
1. unexpected ‘(‘
2. it printed welcome {1..5} times instead repeating it…

help..?

Reply

5 balamurugan August 11, 2010 at 5:24 am

hi manish your both coding are correct… before execute you must give the execution permission for that file… so you try following steps…
1.goto terminal
2. vim simple
3.then write the following code..

for (( c=1; c<=5; c++ ))
do
echo “Welcome $c times…”
done

4.then save and quit
5.chmod 744 simple
6. ./simple

i hope surely it will help you…

Reply

6 Manish Kumar Mishra February 3, 2011 at 5:16 am

It works properly just check it again..

Reply

7 Dr. Stefan Gruenwald April 1, 2011 at 3:53 am

I can help you on 2. — You were not using the Bash 3.0 or higher. Upgrade your bash and it will work.

Reply

8 Niranjan May 10, 2011 at 1:35 am

Hi All,

I have some ‘.gif’ and ‘.jpg’ files in a directory named Pictures in my home directory. I need to write bash script that would create 2 separate html files such as page1.html and page2.html one for gif files and the other for jpg files. And when i execute the script i need to have the html files in the Pictures directory and should have the contents as follows:

filename.jpg
filename.jpg

filename.gif
filename.gif

Please help me out. Thanks in advance

Reply

9 krist0ph3r May 30, 2011 at 2:37 pm

your problem is very easy to solve using the examples on the page.

in addition to the for loops, you will need to use the echo command, the redirection operator >> and a basic knowledge of html.

your script should do:

1. create a html file with the header, opening body tags etc.
2. have a loop for all jpg files
2.1 inside the loop, print one line with the html code for an image, using the image’s filename
3. close the loop, add closing html tags
4-6. same as 1-3, but for .gif instead of .jpg

Reply

10 lascost December 6, 2008 at 6:15 pm

i tried the last example but i seen dint work

#!/bin/bash
set -x
FILLES="$@"
CP=$(which cp)
for f in $FILES
do
        if [ -f ${f}.bak ]
        then
                echo "skiping $f file"
                continue # read netxt file and skip cp command
        fi
        $CP $f $f.bak
done

i would like know where is the error

Reply

11 nixCraft December 13, 2008 at 4:56 pm

Replace

FILLES="$@"

With

FILES="$@"

Reply

12 lo2y January 28, 2009 at 10:10 am

hi guys . can any one help me . i need a script to check the file /var/log/messages every 10 minutes .and if its has the following log :
ext3_orphan_cleanup: deleting unreferenced

to apply the following command
sendsms to wut ever .

thnx alot

Reply

13 swatkat February 12, 2009 at 2:06 pm

i would like to breakk a csv file depending upon two criteria.

1. SIngle file should not be more than 100 lines
2. The third column if has same value on the 100th line as that of the 101th line, the complete line should be included in the 2nd file.
so., now, 1st file will have 99 lines and 2nd file will have 100 lines, ifthe above 2nd condition does not repeats.,

Reply

14 archana March 16, 2009 at 11:53 pm

for file in $(ls 0902*0010202.TLG); do
day=$(echo $file | cut -c 1-6)
grep ^203 $file | cut -d, -f3 | sort | uniq -c | while read line; do
cnt=$(echo $line | cut -d” ” -f1)
acct=$(echo $line | cut -d” ” -f2)
echo “Date 20${day} Account ${acct} had ${cnt} 203′s” >> Feb_report.txt
done
done

when i run it it gives me a syntax error
ins@ARTMGA01> ./arc.sh
./arc.sh: syntax error at line 4: `$’ unexpected

could you help

Reply

15 Navneet April 20, 2009 at 10:57 am

Good examples!! easily understood

Reply

16 firmit June 7, 2009 at 5:06 pm

Hi

How do I read line by line in a file, and use these in a loop? I have a file I read in (cmd max_cpu):
firefox 15
conky 1
cmds=$(cat file)
But $cmds now consist of n items, all being “equal” – it does not split on each line to a new array. I expected that by looping over $cmds, I’d get a 2D array…. I did not.

Otherwise, excellent tutorial!

Reply

17 nixCraft June 7, 2009 at 5:56 pm

Try:

FILE=/etc/passwd
while read line
do
	# store field 1
	F1=$(echo $line|cut -d$FS -f1)
	# store field 2
	F2=$(echo $line|cut -d$FS -f6)
	# store field
	F3=$(echo $line|cut -d$FS -f7)
	echo "User \"$F1\" home directory is $F2 and login shell is $F3"
done < $FILE

Reply

18 Andi Reinbrech November 18, 2010 at 7:42 pm

I know this is an ancient thread, but thought this trick might be helpful to someone:

For the above example with all the cuts, simply do

set `echo $line`

This will split line into positional parameters and you can after the set simply say

F1=$1; F2=$2; F3=$3

I used this a lot many years ago on solaris with “set `date`”, it neatly splits the whole date string into variables and saves lots of messy cutting :-)

… no, you can’t change the FS, if it’s not space, you can’t use this method

Reply

19 Dana Good February 13, 2012 at 3:05 am

This *is* helpful – good knowledge to have. I’ll go give it a try.

Thanks!

Reply

20 Alvin August 17, 2010 at 12:09 pm

Hi Vivek,
Please help I was trying to use your code in a script similar issue trying to use a csv file with three columns (loginname,surname,firname) as input for a file that will be executed
fileA
loginN,ssn,ffn
ab1pp1,ab1,pp1
bb1oo1,bb1,oo1
cc1qq1,cc1,qq1

#file to be changed cmdch.sh
echo “your login name is $loginn, your surname is $ssn and your firname $ffn”

Program
#!/bin/bash

LINNUM=4
while read LINE;
do
LINNUM=`expr $LINENUM + 1`
done < smallops.csv

FILE=fileA.csv
while read LINE;
do
#store field 1
F1=$(echo $line|cut -d$FS -f1)
#store field 2
F2=$(echo $line|cut -d$FS -f6)
#store field 3
F3=$(echo $line|cut -d$FS -f7)
sed '{$LINNUM s/lgn/$F1/g; $LINNUM s/ssn/$F2/g; $LINNUM s/ffn/$F3/g; }' -i smallops.csv
done < g.csv

echo

Reply

21 Philippe Petrinko August 17, 2010 at 2:04 pm

@Alvin

First, this code won’t work at least because it does not input fileA.csv as intended.

Second, to debug, try to break down this program, piece by piece.

For instance, it could read input more easily from fileA.csv this way:
(I have not included your [sed] instruction yet. Go step by step)

# backup current IFS (Internal File Separator)
IFS_backup="${IFS}"
# change IFS to directly read input file into 3 variables a,b,c
IFS=","
while read a b c
do
echo "LOGIN:$a LASTNAME:$b FIRSTNAME:$c"
done < fileA.csv
# restore IFS
IFS="${IFS_backup}"

Third, try to explain what you would like to do with you [sed] instruction.

Why do you start your LINENUM at 4?

What is the content of your smallops.csv? What is it for?

By the way, your [sed] instruction seems to contain a mispelled search pattern, for first field $F1, should’nt it be [loginn] instead of [lgn] ?

You mention only ONE csv file, but your code contains: fileA.csv, smallops.csv, and g.csv ? What are they? Typos ? Errors in your code?

Reply

22 Philippe Petrinko August 17, 2010 at 2:05 pm

Sorry, (I forgot a HTML code TAG) Complete code is:

# backup current IFS (Internal File Separator)
IFS_backup="${IFS}"
# change IFS to directly read input file into 3 variables a,b,c
IFS=","
while read a b c
do
echo "LOGIN:$a LASTNAME:$b FIRSTNAME:$c"
done < fileA.csv
# restore IFS
IFS="${IFS_backup}"

Reply

23 firmit June 7, 2009 at 6:03 pm

Excellent! Thanks Vivek.

Reply

24 Peko July 16, 2009 at 6:11 pm

Hi Vivek,
Thanks for this a useful topic.

IMNSHO, there may be something to modify here
=======================
Latest bash version 3.0+ has inbuilt support for setting up a step value:

#!/bin/bash
for i in {1..5}
=======================
1) The increment feature seems to belong to the version 4 of bash.
Reference: http://bash-hackers.org/wiki/doku.php/syntax/expansion/brace
Accordingly, my bash v3.2 does not include this feature.

BTW, where did you read that it was 3.0+ ?
(I ask because you may know some good website of interest on the subject).

2) The syntax is {from..to..step} where from, to, step are 3 integers.
You code is missing the increment.

Note that GNU Bash documentation may be bugged at this time,
because on GNU Bash manual, you will find the syntax {x..y[incr]}
which may be a typo. (missing the second “..” between y and increment).

see http://www.gnu.org/software/bash/manual/bashref.html#Brace-Expansion

The Bash Hackers page
again, see http://bash-hackers.org/wiki/doku.php/syntax/expansion/brace
seeems to be more accurate,
but who knows ? Anyway, at least one of them may be right… ;-)

Keep on the good work of your own,
Thanks a million.

– Peko

Reply

25 nixCraft July 16, 2009 at 6:31 pm

@ Peko,

Thanks for pointing out ranges vs step value. I’ve updated the FAQ.

Reply

26 Peko July 16, 2009 at 8:04 pm

Yes.

But you mispelled the syntax with an extra dot “.” after “START’
not {START…END..INCREMENT}
but {START..END..INCREMENT}

;-)
– Peko

Reply

27 Michal Kaut July 22, 2009 at 6:12 am

Hello,

is there a simple way to control the number formatting? I use several computers, some of which have non-US settings with comma as a decimal point. This means that
for x in $(seq 0 0.1 1) gives 0 0.1 0.2 … 1 one some machines and 0 0,1 0,2 … 1 on other.
Is there a way to force the first variant, regardless of the language settings? Can I, for example, set the keyboard to US inside the script? Or perhaps some alternative to $x that would convert commas to points?
(I am sending these as parameters to another code and it won’t accept numbers with commas…)

The best thing I could think of is adding x=`echo $x | sed s/,/./` as a first line inside the loop, but there should be a better solution? (Interestingly, the sed command does not seem to be upset by me rewriting its variable.)

Thanks,
Michal

Reply

28 Peko July 22, 2009 at 7:27 am

To Michal Kaut:

Hi Michal,

Such output format is configured through LOCALE settings.

I tried :

export LC_CTYPE=”en_EN.UTF-8″; seq 0 0.1 1

and it works as desired.

You just have to find the exact value for LC_CTYPE that fits to your systems and your needs.

Peko

Reply

29 Peko July 22, 2009 at 2:29 pm

To Michal Kaus [2]

Ooops – ;-)
Instead of LC_CTYPE,
LC_NUMERIC should be more appropriate
(Although LC_CTYPE is actually yielding to the same result – I tested both)

By the way, Vivek has already documented the matter : http://www.cyberciti.biz/tips/linux-find-supportable-character-sets.html

– Peko

Reply

30 VIKAS July 22, 2009 at 3:58 pm

Excellent stuff… keep up the good work.

Reply

31 Brad Landis October 26, 2009 at 3:26 pm

Comment 12 was really helpful. I was trying to split up a log file by date, such as
logfile.20091026 , without having to use grep a million times. I’m kind of disappointed I couldn’t find a one-liner to do so, but I will take what I can get :).

Reply

32 nixCraft October 26, 2009 at 4:47 pm

@Brad,

Try this without grep or cut using bash parameter expansion :

file="logfile.20091026"
log="${file%%.*}"
date="${file##*.}"
echo $log
echo $date

HTH

Reply

33 Philippe Petrinko October 26, 2009 at 4:52 pm

H i vivek,

Just wondering why you don’t amend the typo I pointed out:
<>

Reply

34 Brad Landis October 26, 2009 at 4:53 pm

I think you misunderstood. I’m going line by line, and converting the dates at the beginning of the line, such as “Sep 12″, and copying that line from logfile to logfile.20090912.

My script is really slow though, with the conversion of the month name to a number. I’ve tried using the date command, and my own function, and both take 7 seconds to process 10,000 lines. It doesn’t seem like a long time, but I’ve got a lot of log files to process on multiple machines.

I don’t guess you’d know a faster trick, would you?

Reply

35 nixCraft October 26, 2009 at 6:37 pm

@Brad, yes, I did misunderstood your post. If I were you I will try out awk.

@Philippe,
Thanks for the heads up. The faq has been updated.

Reply

36 TheBonsai October 30, 2009 at 6:13 am

@Peko:

(I’m the operator of bash-hackers.org/wiki, that’s why I found this page):

Regarding Bash documentation for brace expansion (increment syntax), actually I’m right and the documentation is wrong (a rare situation!). I reported it to the list.

Reply

37 Philippe Petrinko October 30, 2009 at 8:35 am

To Vivek:
Regarding your last example, that is : running a loop through arguments given to the script on the command line, there is a simplier way of doing this:
# instead of:
# FILES=”$@”
# for f in $FILES

# use the following syntax
for arg
do
# whatever you need here – try : echo “$arg”
done

Of course, you can use any variable name, not only “arg”.

To TheBonsai: Welcome Buddy!
Fine! I am happy to see 2 great FOSS web sites now related !

Reply

38 tdurden November 10, 2009 at 9:32 pm

Command line while loop.. Very handy..

Say you wanted to rename all the files in a specific dir..
Create a file with the contents you want to rename
(ls -l | awk ‘{print $9}’ > asdf or something)

Contents of asdf:
file1
file2
file3
file4

cat asdf | while read a ; do mv $a $a.new ; done

ls -l
asdf file1.new file2.new file3.new file4.new

I have used this while command for many things from simply renaming files to formatting and labling new SAN luns..

Reply

39 Dr. Stefan Gruenwald April 1, 2011 at 4:00 am

There are much easier ways to do this – also it works only for extensions. How do you change the middle of the file name or a few characters on the left?

Here is the regular way of what you just did:

for i in *; do mv $i $i.new; done

Reply

40 TheBonsai November 11, 2009 at 7:07 am

There are 2 problems and one optical flaw with your code:

(1) You should use read -r without any variable name given, to use the default $REPLY (due to a specific behaviour of read, see manpage)
(2) You should quote $a
(3) Useless use af cat :)

Reply

41 Philippe Petrinko November 11, 2009 at 11:25 am

To tdurden:

Why would’nt you use

1) either a [for] loop
for old in * ; do mv ${old} ${old}.new; done

2) Either the [rename] command ?
excerpt form “man rename” :

RENAME(1) Perl Programmers Reference Guide RENAME(1)

NAME
rename – renames multiple files

SYNOPSIS
rename [ -v ] [ -n ] [ -f ] perlexpr [ files ]

DESCRIPTION
“rename” renames the filenames supplied according to the rule specified
as the first argument. The perlexpr argument is a Perl expression
which is expected to modify the $_ string in Perl for at least some of
the filenames specified. If a given filename is not modified by the
expression, it will not be renamed. If no filenames are given on the
command line, filenames will be read via standard input.

For example, to rename all files matching “*.bak” to strip the
extension, you might say

rename ‘s/\.bak$//’ *.bak

To translate uppercase names to lower, you’d use

rename ‘y/A-Z/a-z/’ *

– Philippe

Reply

42 TheBonsai November 11, 2009 at 11:49 am

Note for rename(1): There exist two major variants on Linux system. One non-Perl originating in the RedHat area, and one Per, originating in the Debian area.

Reply

43 Sean November 11, 2009 at 2:42 pm

To tdurden:

I would also replace “ls -l | awk ‘{print $9}’” with just “ls”. Otherwise you’ll run into issues with files that have spaces in it. As far as using:

for i in *;
vs
for i in $(ls);

I personally prefer “$(ls)” or “$(find . )”. This provides more control over what files I’m going to be looping through. For instance:

$(ls -F | grep -v “\/$”)
or
$(ls -A)

Reply

44 A.R.Memon December 4, 2012 at 9:58 am

#!/bin/bash
echo “Bash version ${BASH_VERSION}…”
for i in {0..10..2}
do
echo “Welcome to my new script $i times”
done

help ……………………….. :)

Reply

45 Philippe Petrinko November 11, 2009 at 3:03 pm

To Sean:
CMIIAW :

try the following commands:
# touch “file with spaces in name”
# for f in *; do echo “”;done

which shows that there is no need to use [for f in $(ls)] instead of [ for f in *]
Doesn’t it ?

– Philippe

Reply

46 Philippe Petrinko November 11, 2009 at 3:14 pm

Sorry Sean, my last post was truncated,
due to limitations of this form used to post comments. (impossible to use Greater_than and Less_than caracters)

I meant, use the following:
# touch “file with spaces in name”
# for f in *; do echo “:${f}:”;done

:file with spaces in name:

Reply

47 Sean November 11, 2009 at 8:10 pm

Sorry for the confusion, I understand that “for i in *;” will not have any issues with spaces. I was referring to the ls -l | awk ‘{print $9}’ having issues with spaces. The reason I choose to use $(ls) instead of * is for filtering out unwanted files e.g. $(ls -F | grep -v “\/$”)

Reply

48 Philippe Petrinko November 11, 2009 at 9:16 pm

To Sean:

But then, that’s wrong.
[ for f in $(ls -F|grep -v "V$") ]
won’t process appropriately spaces in filename.
Check :
# touch “file with spaces in name”
# for f in $(ls -F|grep -v “V$”); do echo “:${f}:”;done
:file:
:with:
:spaces:
:in:
:name:

Reply

49 Philippe Petrinko November 11, 2009 at 9:18 pm

The best tool to filter files and process them
is [find] piped to [xargs] (with zero-ended filenames)

Reply

50 Philippe Petrinko November 11, 2009 at 9:25 pm

To sean:
But if you want to exclude files from globbing,
[bash] has the [extglob] option.

Let’s say you want to process every file except files ending by a “V”, just type

# for f in !(*V); do echo “:${f}:”;done

Reply

51 Philippe Petrinko November 11, 2009 at 9:27 pm

If you set the shell option extglob, Bash understands some more powerful patterns. Here, a is one or more pattern, separated by the pipe-symbol (|).

?() Matches zero or one occurrence of the given patterns
*() Matches zero or more occurrences of the given patterns
+() Matches one or more occurrences of the given patterns
@() Matches one of the given patterns
!() Matches anything except one of the given patterns

Reply

52 Philippe Petrinko November 11, 2009 at 9:28 pm
53 Narender November 12, 2009 at 6:47 am

I have two files here X.a and y.a Now what i need is i need to substitute CvfdsDisk_sdb/c/d/e in lines of Node CvfsDisk_XXX in the order CvfsDisk_sdb/c/f/g first word of each line of x.a exists. how can i do in shell scripting i can get the first word of each line of X.a using awk /cut but to replace these in y.a i am not getting it … any help here ?
[Raj]$ cat x.a
CvfsDisk_sdb /dev/sdb # host 0 lun 1 sectors 4840746976 sector_size 512 inquiry [AMCC 9550SX-12M DISK 3.08] serial AMCC ZAJBSXJFF92A9D003C6A
CvfsDisk_sdc /dev/sdc # host 0 lun 0 sectors 3906148319 sector_size 512 inquiry [AMCC 9550SX-12M DISK 3.08] serial AMCC ZAJ8MJKFF92A9D001FEC
CvfsDisk_sdf /dev/sdf # host 0 lun 1 sectors 4840746976 sector_size 512 inquiry [AMCC 9550SX-12M DISK 3.08] serial AMCC ZAJBSXJFF92A9D003C6A
CvfsDisk_sdg /dev/sdg # host 0 lun 0 sectors 3906148319 sector_size 512 inquiry [AMCC 9550SX-12M DISK 3.08] serial AMCC ZAJ8MJKFF92A9D001FEC
[naren@Beas dxall]$ cat y.a

[StripeGroup Metafiles]
Metadata Yes
Status UP
Read Enabled
Write Enabled
Journal Yes
StripeBreadth 1280K
Node CvfsDisk_sdb 0

[StripeGroup datafiles1]
Metadata Yes
Status UP
Read Enabled
Write Enabled
StripeBreadth 1024K
Node CvfsDisk_sdc 0

[StripeGroup datafiles2]
Metadata Yes
Status UP
Read Enabled
Write Enabled
StripeBreadth 1280K
Node CvfsDisk_sdd 0

[StripeGroup datafiles3]
Metadata Yes
Status UP
Read Enabled
Write Enabled
StripeBreadth 1024K
Node CvfsDisk_sde 0

Reply

54 nixCraft November 12, 2009 at 1:47 pm

@Narender,

Your post is offtopic. I suggest you use our shell scripting forum for question.

Reply

55 Sean November 12, 2009 at 3:26 pm

To Philippe:

You are right,

# for i in $(ls)

will break up files with spaces if IFS isn’t set to just the newline character. I don’t believe this is consistent across distributions. So the for loop should have

# export IFS=$’\n’

before it. I also use find in for loops when I want to look through the directory contents, but this isn’t always desired. Thanks for the info about extglob, I haven’t done much with extended globbing in bash.

Reply

56 Philippe Petrinko November 12, 2009 at 3:44 pm

To Sean:
Right, the more sharp a knife is, the easier it can cut your fingers…

I mean: There are side-effects to the use of file globbing (like in [ for f in * ] ) , when the globbing expression matches nothing: the globbing expression is not susbtitued.

Then you might want to consider using [ nullglob ] shell extension,
to prevent this.
see: http://www.bash-hackers.org/wiki/doku.php/syntax/expansion/globs#customization

Devil hides in detail ;-)

Reply

57 The_Catalanish December 4, 2009 at 11:22 am

Response to the tip number 12
At thos script, It’s missing the followng line
FS=’:’
in the variables declaration
(you forgit the delimiter field, for the cut command)
:-P
The_Catalanish

Reply

58 Rilif December 10, 2009 at 9:27 pm

#! /usr/bin/ksh
for i in `cat /input`
do
bdf | grep file_system | grep -vE ‘^A|B|C’ | awk ‘{ print $4}’ | while read output;
do
file_system=$(echo $output | awk ‘{ print $1}’ | cut -d’%’ -f1 )
partition=$(echo $output | awk ‘{ print $2 }’ )
if [ $file_system -ge 60 ]; then
echo “don’t run the sync $partition ($file_system%) ”
else
rsync $i
fi
done
done
The problem with the logic I’m having is I do not want the script to exit(as it does now) the loop once the file_system area reaches 60%. I want
it to continue to retest bdf and continue the loop once disk usage drops below 60%.

Any Ideas?

Reply

59 Philippe Petrinko December 11, 2009 at 12:17 pm

To Rilif:

1) I assume you use [ bdf ] on UNIX system – because Linux equivalent is [ df ] – and I cannot be of help because I cannot test your script on my Linux boxes.

2) This seems to be a specific programming debugging problem and out of this topic scope – There may be a better place to post that kind of topic – A programmer forum for instance.

Best regards.

Reply

60 dee December 13, 2009 at 10:27 am

do any one know how to write this script in steps?
as so /// ./ test 10 /// The first argument [1] will ex. to create a multiple users, groups, cn, dn, etc for ldap in one or two scripts but from command line. you would just enter file then the number of atrributes to build.
this is a headache for me since i’m new at this.

Reply

61 Philippe Petrinko December 13, 2009 at 10:54 am

To Dee:

0) The first part of your first sentence is incomprehensible – this may be because text that you entered is altered, it may contain HTML-like syntax that is interpreted by this comment form. (By the way, Vivek Gite would be welcomed to tell us how to prevent this. TIA :-) )

1) LDAP syntax is off-topic.

2) You’ll find appropriate resources in LDAP forums/sites – just find out.

3) We may be in position to help you to build a [for] loop, assuming you do your part of the job by providing the basic LDPA instructions to create user, for instance. Try to create at least a LDAP
object by yourself on the command-line, then provide us the code, and as much as possible further explanation please.

Reply

62 nixCraft December 13, 2009 at 10:58 am

There was no html tag or anything else in comment.

@dee, if you need to attach code use <pre> tags.

Reply

63 Philippe Petrinko December 13, 2009 at 11:44 am

Thanks Vivek – But I am afraid I do not get it right – what does “pre” mean ? (I understand you wrote the “less than” tag, and “greater than” tag – but why “pre” ?

And are you sure these are the only ones two use ?

Reply

64 nixCraft December 13, 2009 at 1:26 pm

@Philippe,

All allowed html tags are displayed below the form itself. It is wordpress that converts those symbol and syntax is

<pre>#!/bin/bash
echo "Hello world!"
</pre>

HTH

Reply

65 dee December 13, 2009 at 1:41 pm

first file make: create_user_idif.sh.txt

#!/bin/bash
echo "dn: $2 $3,o=dmacc,dc=bad1dee,dc=com" > $1
echo "changetype: add" >> $1
echo "objectclass: top" >> $1
echo "objectclass: person" >> $1
echo "objectclass: organizationalPerson" >> $1
echo "objectclass: inetOrgPerson" >> $1
echo "cn: $2 $3" >> $1
echo "givenName: $2" >> $1
echo "sn: $3" >> $1
echo "uid: ${2:0:1}$3" >> $1
echo "mail: $4" >> $1
___________________________________
mass_user.txt
Scott,brown,sbrown@bad4dee.com\n
karla,smith,ssmith@bad4dee.com\n
sam,goodie,sgoodie@bad4dee.com\n
marge,jones,mjones@bad4dee.com
____________________________________
mass_add.sh.txt
#!/bin/bash
u1=`cat $1 | tr -d '\n'`
#u2=`echo $u1 | tr \| '\n'`
u2=`echo $u1 | sed s/\|/\\n/`
echo "DBG"
echo -e $u1
echo "DBG"
echo -e $u1 | ( IFS=, ; while read fname lname mail;
do
echo -e "Creating $fname $lname\n\n"

./create_user_ldif.sh.txt ./user.ldif $fname $lname $mail

cat ./user.ldif
echo -e "\n\n"
done )

_____________________________________
./mass_add.sh.txt mass_user.txt
This ex: will pull from a list but same out come i do not know how to write another script to pull the attributes i need from the command line like ./test 100 and that command will pull only a 100 users id’s from idif.txt out of 1000 generated.

This next samples of code will file in the attributes for you.
first file make: create_user_idif.sh.txt

#!/bin/bash
echo "dn: $2 $3,o=dmacc,dc=bad1dee,dc=com" > $1
echo "changetype: add" >> $1
echo "objectclass: top" >> $1
echo "objectclass: person" >> $1
echo "objectclass: organizationalPerson" >> $1
echo "objectclass: inetOrgPerson" >> $1
echo "cn: $2 $3" >> $1
echo "givenName: $2" >> $1
echo "sn: $3" >> $1
echo "uid: ${2:0:1}$3" >> $1
echo "mail: $4" >> $1

____________________________________________________________
Then:

mass_add.sh.txt

#!/bin/bash
u1=`cat $1 | tr -d '\n'`
#u2=`echo $u1 | tr \| '\n'`
u2=`echo $u1 | sed s/\|/\\n/`
echo "DBG"
echo -e $u1
echo "DBG"
echo -e $u1 | ( IFS=, ; while read fname lname mail;
do
echo -e "Creating $fname $lname\n\n"
./create_user_ldif.sh.txt ./user.ldif $fname $lname $mail
cat ./user.ldif
echo -e "\n\n"
done )

_____________________________________________________________________
last:
adduserfile1.txt

user0,boo,us1@bad1dee.com\n
user1,boo,us1@bad1dee.com\n
user2,boo,us1@bad1dee.com\n
user3,boo,us1@bad1dee.com\n
user4,boo,us1@bad1dee.com\n
user5,boo,us1@bad1dee.com\n
./mass_add.sh.txt adduserfile1.txt

This is what i'm working on now ? i still do not know how to tie in C++ or bash script this code to work with command line, so i can control the out come of created users.

#!/bin/bash
Set objRootDSE = GetObject("LDAP://rootDSE")
Set objContainer = GetObject("LDAP://cn=Users," & _
    objRootDSE.Get("defaultNamingContext"))
For i = 1 To 1000
    Set objLeaf = objContainer.Create("User", "cn=UserNo" & i)
    objLeaf.Put "sAMAccountName", "UserNo" & i
    objLeaf.SetInfo
Next
WScript.Echo "1000 Users created."
example: [root]#./test 10
usid:UserNo1 gid:manager cn:Bob dn:bHarison
UserNo2
UserNo3
^^^^^^^
UserNo10      

This script makes a 1000 users. However i can not control the out come. For example: from the command line I would like it to stop at 100 users by typing in ./test 100. Using agrv [1]. so when I type a number after the file name it will create a list and print that record to the screen. I do not know bash that well as C++ and it is not helping because the char.. are diff...

Edited by admin. Reason: This is not a forum, this is a blog. If you need further help please try our forum @nixcraft.com.

Reply

66 Philippe Petrinko December 13, 2009 at 2:51 pm

To Dee:
1) Man, with a 3-users-sample instead of hundreds, we would have figured out, don’t you think so?

2) Well that’s a start. May be Vivek would like to wipe this post out, and create a new topic: “Of mice, LDAP and loops” ;-) ???

Reply

67 Dominic January 14, 2010 at 10:04 am

There is an interesting difference between the exit value for two different for looping structures (hope this comes out right):
for (( c=1; c<=2; c++ )) do echo -n "inside (( )) loop c is $c, "; done; echo "done (( )) loop c is $c"
for c in {1..2}; do echo -n "inside { } loop c is $c, "; done; echo "done { } loop c is $c"

You see that the first structure does a final increment of c, the second does not. The first is more useful IMO because if you have a conditional break in the for loop, then you can subsequently test the value of $c to see if the for loop was broken or not; with the second structure you can’t know whether the loop was broken on the last iteration or continued to completion.

Reply

68 Dominic January 14, 2010 at 10:09 am

sorry, my previous post would have been clearer if I had shown the output of my code snippet, which is:
inside (( )) loop c is 1, inside (( )) loop c is 2, done (( )) loop c is 3
inside { } loop c is 1, inside { } loop c is 2, done { } loop c is 2

Reply

69 Dmitry March 8, 2010 at 5:47 pm

What if I would like to put everything into one line?

Reply

70 Dominic March 9, 2010 at 5:29 am

Dmitry, please give a little more detail about what you are trying to achieve. You can see from my examples above that there is no problem to put a simple loop on one line. Basically you use semicolons (;) instead of line breaks.

Reply

71 Dmitry March 9, 2010 at 1:56 pm

Dominic,
thaks a lot for your quick answer. You have answered on my question but I’m still having problems. Please take a look at this.
dmitry@elastix-laptop:~/projects_cg/match_delays/source$ for i in $(seq 1 2 20)
> do
> echo “Welcome $i times”
> done
Welcome 1 times
Welcome 3 times
Welcome 5 times
Welcome 7 times
Welcome 9 times
Welcome 11 times
Welcome 13 times
Welcome 15 times
Welcome 17 times
Welcome 19 times
dmitry@elastix-laptop:~/projects_cg/match_delays/source$ for i in $(seq 1 2 20); do; echo “Welcome $i times” ; done
bash: syntax error near unexpected token `;’
dmitry@elastix-laptop:~/projects_cg/match_delays/source$
What am I missing here?

Reply

72 Philippe Petrinko March 9, 2010 at 2:05 pm

@Dmitry
You are missing : Reading The Fantastic Manual. :-)

man bash
...
for name [ in word ] ; do list ; done
...
for (( expr1 ; expr2 ; expr3 )) ; do list ; done
...

There should not be any “;” following the [do].

for i in $(seq 1 2 20); do echo "Welcome $i times" ; done

Good ol’ one: “When any thing goes wrong – (re) Read the manual”

Reply

73 Philippe Petrinko March 9, 2010 at 2:34 pm

@Dmitry

And, again, as stated many times up there, using [seq] is counter productive, because it requires a call to an external program, when you should Keep It Short and Simple, using only bash internals functions:


for ((c=1; c<21; c+=2)); do echo "Welcome $c times" ; done

(and I wonder why Vivek is sticking to that old solution which should be presented only for historical reasons when there was no way of using bash internals.
By the way, this historical recall should be placed only at topic end, and not on top of the topic, which makes newbies sticking to the not-up-to-date technique ;-) )

Reply

74 Dmitry March 9, 2010 at 2:49 pm

Philippe, thank you, all works perfect know.
Here is the partial excuse that I was reading this thread instead of bash manual
http://www.google.es/search?q=bash+for+loop&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-US:official&client=firefox-a

Reply

75 Dominic March 9, 2010 at 3:23 pm

It is strangethat a do/done loop works if there is CR (end of line) after the do, but not if there is a semi-colon. I can see why this was confusing for you Dmitry, because it’s not logical. My guess is that the acceptance of CR after do was added because people wanted to lay out code this way, but the bash coders forgot to allow the semicolon alternative. As Philippe points out, if you follow the manual strictly, it works fine.

Reply

76 Dmitry March 9, 2010 at 3:27 pm

Ok. Thank you again!

Reply

77 nixCraft March 9, 2010 at 5:30 pm

@Philippe,

I’ve just updated the faq and also deleted large data sample posted by dee user.

Reply

78 TheBonsai March 9, 2010 at 5:55 pm

@ Dominic

Yes, it’s not quite intuitive, right. What you mean is the semicolon or the newline as list separator (list as the grammar construct defined in the manual, respectively by ISO9945). After a `do’, a list is expected, but a list isn’t introduced with a list separator. After a `do’, the shell awaits more input, just like after an opening quote character. In interactive mode, it also displays the continuation prompt `PS2′ instead of `PS1′ (it would display `PS1′ for list continuation).

It’s not right that the Bash coders “forgot” it. Inspecting the grammar rules of POSIX XCU Sec. 2.10.2 doesn’t show a special rule here (it would have to be a special exceptional rule that extra allows a semicolon here).

Reply

79 TheBonsai March 9, 2010 at 6:06 pm

Me again.

From all Bourne-like shells I just “tested”, only ZSH seems to support a semicolon as a start of a list (also in the case after the `do’). This is nice, but that’s all. It’s not a bug to not do so.

Reply

80 Dominic March 9, 2010 at 6:54 pm

@ TheBonsai

Interesting. I accept that it is not a bug, but I still think it is confusing. It seems logical to us lesser mortals that in bash semicolon=newline, and in other situations I think this is true, but not here.
# this works
for (( c=1; c<=2; c++ )) do

#any number of blank lines or even comments

echo $c; done
# so does this:
for (( c=1; c<=2; c++ )) do echo $c; done
# but this, where we substitute a semi-colon for the blank line(s) above, doesn't:
for (( c=1; c<=2; c++ )) do; echo $c; done

Reply

81 Philippe Petrinko March 9, 2010 at 11:10 pm

@Dominic
You are missing the point.

If you read our Unix pioneers, you will remember:

- Rule of Optimization: Prototype before polishing. Get it working before you optimize it. [E. Raymond]
see http://en.wikipedia.org/wiki/Unix_philosophy

What’s the point of spending hours to code on one line?
- First, It does not give any optimization, it does not save any execution time. Your code will only be more difficult to read, check, debug.

Defensive programming rules include this: Write one instruction per line.

- Second, You still wanna code all on one line ?
Big deal. The manual gave you the right way.
So stick to it, or leave it, and skip to the next real problem, instead of wasting time and energy pointlessly, my dear Linux enthusiast.

Reply

82 Sean March 9, 2010 at 11:15 pm

I have a comment to add about using the builtin for (( … )) syntax. I would agree the builtin method is cleaner, but from what I’ve noticed with other builtin functionality, I had to check the speed advantage for myself. I wrote the following files:

builtin_count.sh:

#!/bin/bash
for ((i=1;i<=1000000;i++))
do
echo "Output $i"
done

seq_count.sh:

#!/bin/bash
for i in $(seq 1 1000000)
do
echo "Output $i"
done

And here were the results that I got:
time ./builtin_count.sh
real 0m22.122s
user 0m18.329s
sys 0m3.166s

time ./seq_count.sh
real 0m19.590s
user 0m15.326s
sys 0m2.503s

The performance increase isn’t too significant, especially when you are probably going to be doing something a little more interesting inside of the for loop, but it does show that builtin commands are not necessarily faster.

Reply

83 Andi Reinbrech November 18, 2010 at 8:35 pm

The reason why the external seq is faster, is because it is executed only once, and returns a huge splurb of space separated integers which need no further processing, apart from the for loop advancing to the next one for the variable substitution.

The internal loop is a nice and clean/readable construct, but it has a lot of overhead. The check expression is re-evaluated on every iteration, and a variable on the interpreter’s heap gets incremented, possibly checked for overflow etc. etc.

Note that the check expression cannot be simplified or internally optimised by the interpreter because the value may change inside the loop’s body (yes, there are cases where you’d want to do this, however rare and stupid they may seem), hence the variables are volatile and get re-evaluted.

I.e. botom line, the internal one has more overhead, the “seq” version is equivalent to either having 1000000 integers inside the script (hard coded), or reading once from a text file with 1000000 integers with a cat. Point being that it gets executed only once and becomes static.

OK, blah blah fishpaste, past my bed time :-)

Cheers,
Andi

Reply

84 Philippe Petrinko November 18, 2010 at 9:06 pm

@Andi

> OK, blah blah fishpaste, past my bed time :-)

Interesting comments anyway!

Reply

85 Philippe Petrinko March 10, 2010 at 9:13 am

@Sean
1) Again, when your only programming concern that last will be the optimization of your loops, you could invest time into such timings.
This would be when there are no other bugs in your code, which I wish you to enjoy ASAP.

2) But then you may find that the real problem/bottleneck is not a for loop.
As Rob Pike said : “Measure. Do not tune for speed until your performance analysis tool tells you which part of the code overwhelms the rest.” [ http://en.wikipedia.org/wiki/Unix_philosophy#Pike:_Notes_on_Programming_in_C ]

3) I agree with you when you say that your code is not relevant as a timing of real-sized programs.

4) Relating to your benchmark “builtin vs. external” commands
test the builtin [ for i in {1..1000000} ]
and you will see that it is very close to [for i in $(seq 1 1000000)]

Reply

86 2012 Doom Day March 10, 2010 at 7:01 pm

You are missing : Reading The Fantastic Manual. :-)

A.. uh.. ?
The lack of examples in the bash man page is the main reason to *avoid* man page. Everyone, knowns how to use them. Syntax is all good, only if you know the bash and UNIX in and out. This is the main reason why most people purchase bash and shell scripting books from O’reilly or Amazon. Rest of freeloaders depends upon Google and site like this to get information quickly. man pages are for gurus; for all new user examples are the best way to get started.

Reply

87 Philippe Petrinko March 10, 2010 at 9:31 pm

@ 2012DD
I agree – I use any of resource I need, man page, –help page, info page, web pages, books.
The thing is: you should try to read man page once.
And actually, if he did, he would have find the syntax.
Vivek’s web site and contributions do not prevent you of reading the “fantastic” manual.

And, as usual, the ones that issue the bitter critics will not move a finger to enhance the manual.

You say man page lacks good examples?
Did you ever try to contribute to any man pages ?
Hope you did.

Reply

88 TheBonsai March 11, 2010 at 11:30 am

The Bash manual page isn’t meant as tutorial. It’s a syntax, grammar and behaviour reference. However, it contains the knowledge to explain why a for loop using `seq’, one using brace expansion and one using builtin arithmetics have the performance relations they actually show when you execute them. The point is to make a relation between abstract descriptions and real execution behaviour. If such things really count, however, I suspect you code in the wrong language.

Reply

89 Philippe Petrinko March 13, 2010 at 9:50 pm

@Bonsai
> The Bash manual page isn’t meant as tutorial. It’s a syntax, grammar and behaviour reference.

Actually, no. A man page can/should contain a EXAMPLE section.
check : http://en.wikipedia.org/wiki/Man_page#Layout

I am pretty happy when the Example section is relevant. And when you want some more, nothing prevents you to try to add new examples… Let’s contribute!

Reply

90 TheBonsai March 14, 2010 at 12:34 am

@Philippe

How many examples would you place there, to show the “common usage of Bash”? I agree that manpages usually should contain examples, but I think this would be too much. Huge manpages (huge because they describe a huge set of functionality) contain very small/no examples.

Reply

91 2012 Doom Day March 14, 2010 at 5:39 am

@Philippe,

I am learning Bash scripting and that is why I’m here and its wiki. Take a look at Solaris UNIX man page, most of them have good set of examples. Another candidate is FreeBSD, they also have good set of examples. Just discovered that our HP-UX came with printed “Posix Born Shell Scripting Manual”.

>Did you ever try to contribute to any man pages ?

No, I’m learning and if I *contribute* anything, I’m dam sure most shell scripting gurus will eat me alive, as I’m not experienced coder. Once I tried to help someone on comp.unix.shell and most of other people on list were so mean to my code that I almost stopped visiting comp.unix.shell. *Beep* those bastards!

Edited by admin

Reply

92 TheBonsai March 19, 2010 at 5:21 am

@2012 Doom Day

Contribution is more than just knowing code. Alone the fact that you write here is a contribution (to the community). Translations, documentation, searching bugs, helping others, sharing expiriences, …

Reply

93 Philippe Petrinko March 19, 2010 at 8:35 am

Quite right, Bonsai,
and you can contribute to Wikibooks, Wikipedia, and Vivek’s Wiki using the books you own and all you have learnt, no one will ever prevent you of doing so, assuming you improve the content. Don’t be shy and be confident on your capacities.
See, I wrote loads of questionable comments and Vivek has not banned me [yet] ;-).

Reply

94 Chris Cheltenham April 2, 2010 at 7:33 pm

Do you know why this doesn’t output anything?

#!/bin/bash
for i in $(cat /$HOME/client_list.txt)
do
echo $i > /home/$i_file.log
done

Reply

95 Philippe Petrinko April 2, 2010 at 8:47 pm

@Chris C

If you want a good explanation, first try to ask a good question and explain:
1a) What you want to do with this program
1b) what your program is supposed to do.

Second:
Mainly, it will fail because there will not be variable expansion $i_file.log should be ${i}_file.log
Read again about variable expansion.

And:
2a) in your ” cat “, there should not be a leading slash before $HOME (because $HOME contains a leading slash) – anyway this wont prevent it from working – but may come to bugs someday.

2c) if a line in client_list.txt does contains spaces, what do you think this would do? Use quotes.

2d) If it still fails, check your permissions to create and overwrite a file in “/home” directory

2e) As said many times, you do not need to use ” for + cat ” to use the content of a file. Just use a while loop :

while read i
echo ${i}
done < $HOME/client_list.txt

Reply

96 Moihan May 9, 2010 at 11:36 am

Sir,
This is a new post. From a file in unix server with a column of temperature, I want to extract a number if it goes greater than 100. Normally it will be in 60 – 80 range. Can u suggest a bash script?

Reply

97 nixCraft May 9, 2010 at 1:00 pm

Use our shell scripting forum for questions.

Reply

98 Anthony Thyssen June 4, 2010 at 6:53 am

The {1..10} syntax is pretty usless as you can use a variable with it!

limit=10
echo {1..${limit}}
{1..10}

You need to eval it to get it to work!

limit=10
eval "echo {1..${limit}}"
1 2 3 4 5 6 7 8 9 10

‘seq’ is not avilable on ALL system (MacOSX for example)
and BASH is not available on all systems either.

You are better off either using the old while-expr method for computer compatiblity!

   limit=10; n=1;
   while [ $n -le 10 ]; do
     echo $n;
     n=`expr $n + 1`;
   done

Alternativally use a seq() function replacement…

 # seq_count 10
seq_count() {
  i=1; while [ $i -le $1 ]; do echo $i; i=`expr $i + 1`; done
}
# simple_seq 1 2 10
simple_seq() {
  i=$1; while [ $i -le $3 ]; do echo $i; i=`expr $i + $2`; done
}
seq_integer() {
    if [ "X$1" = "X-f" ]
    then format="$2"; shift; shift
    else format="%d"
    fi
    case $# in
    1) i=1 inc=1 end=$1 ;;
    2) i=$1 inc=1 end=$2 ;;
    *) i=$1 inc=$2 end=$3 ;;
    esac
    while [ $i -le $end ]; do
      printf "$format\n" $i;
      i=`expr $i + $inc`;
    done
  }

Edited: by Admin – added code tags.

Reply

99 Philippe Petrinko June 4, 2010 at 7:34 am

@Anthony.
Quite right – braces {start..end..step} might not be the best thing in bash.

Nevertheless, I still stick to the old C-like synxtax in a for loop, which does accept variable arguments, such as:


xstart=10;xend=20;xstep=2
for (( x = $xstart; x <= $xend; x += $xstep)); do echo $x;done
10
12
14
16
18
20

I don’t know much of this FOR loop syntax portability, functions you suggest may be the best thing to use for portability concern. (I have to read POSIX reference again :-) )

Reply

100 TheBonsai June 4, 2010 at 9:57 am

The Bash C-style for loop was taken from KSH93, thus I guess it’s at least portable towards Korn and Z.

The seq-function above could use i=$((i + inc)), if only POSIX matters. expr is obsolete for those things, even in POSIX.

Reply

101 Philippe Petrinko June 4, 2010 at 10:15 am
102 Anthony Thyssen June 6, 2010 at 7:18 am

TheBonsai wrote…. “The seq-function above could use i=$((i + inc)), if only POSIX matters. expr is obsolete for those things, even in POSIX.”

I am not certain it is in Posix. It was NOT part of the original Bourne Shell, and on some machines, I deal with Bourne Shell. Not Ksh, Bash, or anything else.

Bourne Shell syntax works everywhere! But as ‘expr’ is a builtin in more modern shells, then it is not a big loss or slow down.

This is especially important if writing a replacement command, such as for “seq” where you want your “just-paste-it-in” function to work as widely as possible.

I have been shell programming pretty well all the time since 1988, so I know what I am talking about! Believe me.

MacOSX has in this regard been the worse, and a very big backward step in UNIX compatibility. 2 year after it came out, its shell still did not even understand most of the normal ‘test’ functions. A major pain to write shells scripts that need to also work on this system.

Reply

103 TheBonsai June 6, 2010 at 12:35 pm

Yea, the question was if it’s POSIX, not if it’s 100% portable (which is a difference). The POSIX base more or less is a subset of the Korn features (88, 93), pure Bourne is something “else”, I know. Real portability, which means a program can go wherever UNIX went, only in C ;)

Reply

104 TheBonsai June 4, 2010 at 5:57 pm

I’ve read the standard ;-)

Reply

105 Phil Goetz June 17, 2010 at 5:38 pm

That {1 .. N} syntax doesn’t work with current Linux bash.

$for r in {1 .. 15}; do echo $r; done
{1
..
15}

Reply

106 nixCraft June 17, 2010 at 5:57 pm

Yes, it does works, you need bash version 3.0 or up. Just tested with “GNU bash, version 4.1.5(1)-release (i486-pc-linux-gnu)”. You need to remove white space between 1 and 15, try:

for r in {1..15}; do echo $r; done

Reply

107 Anthony Thyssen June 18, 2010 at 12:49 am

I gave some ‘seq’ alternatives, some simple, some more complex, mostly using shell built-ins only, depending on you needs.

At the very start of the comments “jot” was mentioned as an alternative, though it does not appear to be as wide spread as “seq”. Anyone know if it is on the ‘limited shell suport’ MacOSX?

There are also however some other — off the wall — methods of generating a list of number, or just a list for looping ‘N’ times. One of the weirdest ones I came across was using /dev/zero and “dd”!

dd 2>/dev/null if=/dev/zero bs=10 count=1 | tr \ \12 | cat -n | tr -d ‘\40\11′

This gets ’10′ null characters, converts them to line feeds, uses cat to convert them to numbers, and just to clean up, you can optionally delete the tabs and spaces.

As I said real odd ball.

Like I often say…
There are lots of ways to skin a cat, and what method you use depends
on what you want that skin for, and how messy you like the results!

Reply

108 Philippe Petrinko June 18, 2010 at 7:01 am

Yes,
but the code we see won’t work.
I think it needs some syntax enhancement in your first [ tr ], such as:


dd 2>/dev/null if=/dev/zero bs=10 count=1 | tr '00' '12' | cat -n | tr -d '\40\11'

Reply

109 Philippe Petrinko June 18, 2010 at 7:02 am

Hell ! missed again – I hate that $%@ Wordpress text entry

Reply

110 Paddy July 20, 2010 at 8:54 pm

Hi guys Ive been reading this thread cos i need some advice on a script. I need to rename some (lots) of files in a directory. they are named..
file_name.001.01
file_name.002.01
file_name.003.01… etc

How can I change the names of the files to remove the ‘.01′ at the end of each filename? Im useing Ubuntu Lynx…. Ive been playing with a few examples from this thread, but cant seem to make it work. Any help is appreciated

Reply

111 nixCraft July 20, 2010 at 9:12 pm

Try rename command:

rename 's/\.01$//' *.01

If you must use for:

for i in *.01; do mv "$i" "${i%%.01}"; done

HTH

Reply

112 Dr. Stefan Gruenwald April 1, 2011 at 4:53 am

In general, if you want to add a suffix to your files, do this (.txt in this example):

ls
file1 file2 file3

for i in *; do mv $i $i.txt; done

ls
file1.txt file2.txt file3.txt

If you want to take it back off (.txt in this example again)

for i in *.txt; do mv $i ${i%.*}; done

ls
file1 file2 file3

Reply

113 maxy-millianne January 24, 2012 at 6:18 am

Of course, if you want to worry about files with spaces in (and other things?), put quote around the arguments to mv, as in the gp.

Reply

114 Anthony Thyssen January 27, 2012 at 7:48 am

Note that is you want to append a string that does not start with a ‘.’ (for example the string “_info.txt”) then you need to delimit the variable name….

for i in *; do mv “$i” “${i}_info.txt”; done

Reply

115 Sapia July 29, 2010 at 6:20 am

Hi,
I have two files that contain diffrent columns. Both files have matching one column but raw oder is different. I want to combine these two files as below.
File 1: file 2
x 2 7 123 r 3 5 9
y 3 -8 124 y 4 6 20
z 4 -2 34 q 3 5 70
q 5 -9 5 z 5 4 10
r 6 1 6 x 50 3 40

I want to combine each raws considering the common values in the first column
ex:
x 2 7 123 50 3 40
q 5 -9 5 3 5 70

I would be grateful if you could help me with this problem.

Thank you.
Sapia

Reply

116 Philippe Petrinko July 29, 2010 at 11:09 am

@Sapia: As Vivek may say,
“Your post is off-topic. I suggest you use our shell scripting forum for question.”
Go http://nixcraft.com/

Reply

117 borleand September 1, 2010 at 8:23 pm

bash = GPL
GPL = GNU
GNU = Gnu is Not Unix

UNIX logo in your web page = ???

Reply

118 nixCraft September 2, 2010 at 11:45 am

Does it really matters?

Bash runs on both certified UNIX and UNIX like (*BSD & various Linux distros) operating systems.

HTH

Reply

119 Philippe Petrinko September 2, 2010 at 12:10 pm

@Vivek

Quite right – who cares ? Nobody.

I just hope that those guys [borleand] would have better working and helping for FOSS community instead of posting those high-quality ( ;-P ) comments.

Thanks for such a great web site, Vivek.

Reply

120 Anthony Thyssen September 3, 2010 at 2:02 am

Of bigger concern between UNIX, GNU, GPL, Linux, Solaris, MacOSX, or whatever else, is what extra support programs are also available.

I use to use ‘seq’ all the time in shell loops. I don’t any more because ‘seq’ is not available on MacOSX. In fact a lot of simple and what I would have though universal support programs are not available on MacOSX.

Reply

121 naveen September 16, 2010 at 4:53 am

how do i run 100 iteration using bash shell script.. i want to know how long will it take to execute one command(start and end time). I want to keep track which iteration is currently running. i want to log each iteration. I have one automated script i need to run and log it.

for i in 1 2 3
do
command1
done

But i want to know how long it takes to complete one iteration. and writes a log… help me please

Reply

122 Philippe Petrinko September 16, 2010 at 7:15 am

Hi naveen,

1) You should at least read thoroughly this topic. Iteration can be done with this syntax

for x in {start..end}

Look above for explanation.

2) You could use [date] command and output it to a file
- as first command of the iteration (echoing something to mark start)
- as last command of the iteration (echoing something to mark end)

3) Log
What kind of log? To know what? What for?

Reply

123 naveen September 16, 2010 at 3:03 pm

i want to write a output in text file(.txt)(log). I want to have a report saying that test ran for 100 iteration. My question i have one automated script that run 100 test file. i want to know how long it takes to complete the one iteration. And i want to keep a copy of the test result in .txt file.

i know this is to log for one iteration…….
command > log.txt

But when you are running iteration for 100. How wil you log it??. this is my question. Just one automated script (command) but there will be 100 test results for 100 iteration. How wil you log all this???

Reply

124 Philippe Petrinko September 16, 2010 at 3:20 pm

You really seem to be a beginner in shell – the best way to start would be to study Vivek’s Wiki first.
Go http://bash.cyberciti.biz/guide/Main_Page

To answer you iteration question:
1) First write the appropriate [for] loop that calls you commands and show us your code. All the information you need is located above in this page. If you cannot write this loop, you’d better learn [bash] on Vivek’s Wiki and come back when you can write a [for] loop with 100 iteration. Good luck! :)

Reply

125 Anthony Thyssen November 19, 2010 at 3:22 am

The problem with this is that csv files can contain quoted strings. which makes just comma separation usless.. For example
LoginN,ssn,ffn
“ab1,pp1″,”ab1″,”pp1″
“bb1,oo1″,”bb1″,”oo1″
“cc1,qq1″,”cc1″,”qq1″

Reply

126 nixCraft November 19, 2010 at 7:58 am

In that case use ” as delimiter. This can be done with sed or cut or any other shell built-in or utility:

echo '"ab1,pp1","ab1","pp1"' | cut -d '"' -f2
echo '"ab1,pp1","ab1","pp1"' | cut -d '"' -f4
echo '"ab1,pp1","ab1","pp1"' | cut -d '"' -f6

for loop can be used:

for i in {2..6..2}; do echo '"ab1,pp1","ab1","pp1"' | cut -d '"' -f${i}; done

You can skip for and use sed too.

HTH

Reply

127 Anthony Thyssen November 22, 2010 at 1:16 am

It isn’t quite that simple. But then any CSV that is more complex is getting a bit beyond simple shell parsing.

Reply

128 Anthony Thyssen November 22, 2010 at 1:17 am

Nifty solution though, especially without access to a multi-character field separator.

Reply

129 Philippe Petrinko November 22, 2010 at 7:58 am

Here is another way to do it,
- in a generic way, that is without having to know number of internal fields,
- without IFS manipulation
- without external function (only builtin):

Let’s say your CSV file contains 3 fields per record (per line):
“aaa bbb ccc”,”ddd eee fff”,”ggg hhh iii”,”jjj kkk lll”
“mmm nnn ooo”,”ppp qqq rrr”,”sss ttt uuu”,”vvv www xxx”
“yyy zzz 111″,”222 333 444″,”555 666 777″,”888 999 000″

To break it in a one-liner, try:

while read; do record=${REPLY}; echo ${record}|while read -d ","; do echo ${REPLY}; done; done<data

The same code in a script is:

#!/bin/bash
while read
do
record=${REPLY}
echo ${record}|while read -d ,
do
echo ${REPLY}
done
done<data

Reply

130 Philippe Petrinko November 22, 2010 at 8:23 am

And if you want to get rid of double-quotes, use:

one-liner code:
while read; do record=${REPLY}; echo ${record}|while read -d ","; do field="${REPLY#\"}"; field="${field%\"}"; echo ${field}; done; done<data

script code, added of some text to better see record and field breakdown:

#!/bin/bash
while read
do
echo "New record"
record=${REPLY}
echo ${record}|while read -d ,
do
field="${REPLY#\"}"
field="${field%\"}"
echo "Field is :${field}:"
done
done<data

Does it work with your data?

– PP

Reply

131 Philippe Petrinko November 22, 2010 at 9:01 am

Of course, all the above code was assuming that your CSV file is named “data”.

If you want to use anyname with the script, replace:

done<data

With:

done

And then use your script file (named for instance “myScript”) with standard input redirection:

myScript < anyFileNameYouWant

Enjoy!

Reply

132 Philippe Petrinko November 22, 2010 at 11:28 am

well no there is a bug, last field of each record is not read – it needs a workout and may be IFS modification ! After all that’s what it was built for… :O)

Reply

133 Anthony Thyssen November 22, 2010 at 11:31 pm

Another bug is the inner loop is a pipeline, so you can’t assign variables for use later in the script. but you can use ‘<<<' to break the pipeline and avoid the echo.

But this does not help when you have commas within the quotes! Which is why you needed quotes in the first place.

In any case It is a little off topic. Perhaps a new thread for reading CVS files in shell should be created.

Reply

134 Philippe Petrinko November 24, 2010 at 6:29 pm

Anthony,
Would you try this one-liner script on your CSV file?

This one-liner assumes that CSV file named [data] has __every__ field double-quoted.


while read; do r="${REPLY#\"}";echo "${r//\",\"/\"}"|while read -d \";do echo "Field is :${REPLY}:";done;done<data

Here is the same code, but for a script file, not a one-liner tweak.


#!/bin/bash
# script csv01.sh
#
# 1) Usage
# This script reads from standard input
# any CSV with double-quoted data fields
# and breaks down each field on standard output
#
# 2) Within each record (line), _every_ field MUST:
# - Be surrounded by double quotes,
# - and be separated from preceeding field by a comma
# (not the first field of course, no comma before the first field)
#
while read
do
echo "New record" # this is not mandatory-just for explanation
#
#
# store REPLY and remove opening double quote
record="${REPLY#\"}"
#
#
# replace every "," by a single double quote
record=${record//\",\"/\"}
#
#
echo ${record}|while read -d \"
do
# store REPLY into variable "field"
field="${REPLY}"
#
#
echo "Field is :${field}:" # just for explanation
done
done

This script named here [cvs01.sh] must be used so:

cvs01.sh < my-cvs-file-with-doublequotes

Reply

135 Philippe Petrinko November 24, 2010 at 6:35 pm

@Anthony,

By the way, using [REPLY] in the outer loop _and_ the inner loop is not a bug.
As long as you know what you do, this is not problem, you just have to store [REPLY] value conveniently, as this script shows.

:-P

Reply

136 James Wilkinson December 14, 2010 at 10:03 am

Thanks for writing this article Vivek – it is very useful. In particular, I didn’t know about bash’s built-in ‘help’ command and was getting frustrated with the lack of detail in ‘man for’

Reply

137 Sammeta December 15, 2010 at 4:48 pm

Hi i need help in sorting some of the raw data actually on the unix machine.

The raw data is some thing like this:

TOP;0004796;T0698;3,8;2,2;1,6;19384;1560;28;0;304;12;0;httpd
TOP;0004796;T0699;0,2;0,1;0,1;12984;1524;28;0;304;1;0;httpd
TOP;0004806;T0145;40,3;38,8;1,5;427904;32648;8;0;1000;62;0;java
TOP;0004806;T0146;30,6;29,7;0,8;433152;37684;8;0;1036;29;0;java
TOP;0004806;T0147;16,8;16,3;0,6;433808;39152;8;0;1048;11;0;java
TOP;0004806;T0148;14,2;13,4;0,7;434132;40300;8;0;1056;11;0;java
TOP;0004806;T0149;13,1;12,5;0,6;434196;41644;8;0;1080;7;0;java
TOP;0004806;T0150;18,1;17,3;0,8;434276;42556;8;0;1092;18;0;java
TOP;0004806;T0151;8,3;7,8;0,5;434316;42624;8;0;1096;1;0;java
TOP;0004806;T0152;4,4;4,1;0,4;434328;42916;8;0;1096;9;0;java
TOP;0004806;T0153;4,7;4,3;0,4;434352;43204;8;0;1104;9;0;java
TOP;0004806;T0154;3,1;2,8;0,3;434404;43260;8;0;1104;1;0;java
...
VM;T1439;-1;8;-1;-1;-1;41;0;0;0;7346;0;23;0;2312;2312;0;0;0;0;0;0;0;21;21;0;12;-2633;0;0;0;356;-3;0;0;0;0;0
VM;T1440;-1;6;-1;-1;-1;8893;0;0;0;10808;0;12178;-36;865179;-34821;0;0;8895;0;0;0;0;11758;11758;0;-21;2613;0;0;0;-752;3;0;2;0;0;7
ZZZZ;T0001;00:00:02;18-NOV-2010
ZZZZ;T0002;00:01:02;18-NOV-2010
ZZZZ;T0003;00:02:02;18-NOV-2010
....
ZZZZ;T0140;02:19:04;18-NOV-2010
ZZZZ;T0141;02:20:04;18-NOV-2010
ZZZZ;T0142;02:21:04;18-NOV-2010
ZZZZ;T0143;02:22:04;18-NOV-2010
ZZZZ;T0144;02:23:04;18-NOV-2010
ZZZZ;T0145;02:24:04;18-NOV-2010
ZZZZ;T0146;02:25:04;18-NOV-2010
ZZZZ;T0147;02:26:04;18-NOV-2010

So now it has to check for the time stamp T0145 is the timestamp in the below code :

TOP;0004806;T0145;40,3;38,8;1,5;427904;32648;8;0;1000;62;0;java< ?pre>
now this has to be mapped with the same time stamp in ZZZZ.......Which is some thing like this from above.
ZZZZ;T0145;02:24:04;18-NOV-2010

so finally the output should look like this for every occurance the T value:

TOP;0004806;T0145;40,3;38,8;1,5;427904;32648;8;0;1000;62;0;java ZZZZ;T0145;02:24:04;18-NOV-2010,

Reply

138 nixCraft December 15, 2010 at 6:43 pm

See sort command man page or our sort tutorial related FAQs:
Linux / UNIX Shell: Sort Date
Linux / UNIX Shell: Sort IP Address

Reply

139 Sammeta December 16, 2010 at 3:52 pm

Hi Vivek,

Thanks for your reply,

but this is just not only sorting but recording on of the value in the above code and then match this with other lines in the code .then display both together.

Reply

140 Philippe Petrinko December 16, 2010 at 6:10 pm

@Sammeta,

Just trying to help, and not to being rude or anything:
We are not willing to do your work,
anyone would help people who really try to help themselves first,
which you may have been trying already, I suppose.

So, would you either submit a first version of your code,
or at least a main Algorithm you could think of?

You may want to read first [awk] or [join] unix utilities man pages that you could find anywhere.

–P

Reply

141 mohan January 5, 2011 at 12:29 am

what does FILES=”@” do? Please explain i am new to unix.

Reply

142 nixCraft January 5, 2011 at 7:03 am
143 John January 5, 2011 at 1:24 am

One good reason to use seq instead of the {start..end..increment} idiom:

#! /bin/bash
START=1
END=10
INCR=2

echo — use seq —
for x in $(seq $START $INCR $END)
do
echo $x
done
echo — bash idiom —
for x in {$START..$END..$INCR}
do
echo $x
done

———————–
— use seq —
1
3
5
7
9
— bash idiom —
{1..10..2}

Reply

144 TheBonsai January 5, 2011 at 7:09 am

Wrong conclusion. This is not a reason for the seq idiom, it’s a reason to use arithmetically driven (C-like) for loops.

START=1
END=10
INCR=2
for ((x = START; x <= END; x += INCR)); do
  echo $x
done

http://wiki.bash-hackers.org/syntax/ccmd/c_for

Reply

145 Philippe Petrinko February 3, 2011 at 9:09 am

@Bonsai:

The example you gave is weird regarding shell variable usage: It works, but I thought it should not work!
Within the for (( )) instruction, you omitted “$” sign to allow variable expansion, but it works! That looks very strange to me.

I wrote in an example previously (look above):

xstart=1;xend=10;xstep=2
for (( x = $xstart; x <= $xend; x += $xstep)); do echo $x;done

In your example, you wrote [for(( ))] without “$” :

for ((x = START; x <= END; x += INCR))

I am astonished that for(()) works both with and without “$” for variable expansion!

xstart=1;xend=10;xstep=2
for (( x = xstart; x <= xend; x += xstep)); do echo $x;done

Does anyone know why?

–PP

Reply

146 nixCraft February 3, 2011 at 12:15 pm

I think it is ksh93 compatibility feature; so “START / END / INCR” will work with no “$”. See ksh93 man page.

Reply

147 Philippe Petrinko February 3, 2011 at 1:56 pm

Sorry, I cannot find any evidence of such syntax on khs93 man page – and I use Bash shell.
On what URI + what chapter do you think there is an explanation of such behavior?

Reply

148 TheBonsai February 3, 2011 at 5:01 pm

It’s not KSH (or at least not KSH-unique). It’s how arithmetic environments (of any kind) in Bash work.

Also it’s related to what POSIX specifies for the environment inside arithmetic expansion (the only a. environment POSIX knows):

If the shell variable x contains a value that forms a valid integer constant, then the arithmetic expansions “$((x))” and “$(($x))” shall return the same value.

Reply

149 nixCraft February 3, 2011 at 5:29 pm

I think it was mentioned in TLDP.org’s advanced bash guide.

Reply

150 Philippe Petrinko February 3, 2011 at 9:05 pm
151 toto February 16, 2011 at 5:01 am

hi,
good tutorial.
thank you

Reply

152 Víðir Valberg Guðmundsson March 8, 2011 at 3:22 am

I might be going out on a limb due to a bad case of TL;DR, but I noticed the seq warning.

Correct me if I’m wrong but using for instance

for i in $(seq -w 1 1 20); do echo $i; done

is in my oppinion quite an useful way of using seq in bash, at least when you want leading zeros before 1-9 and not from 10<

Or is there a better way of doing this all "bashy"? ;)

-víðir

Reply

153 TheBonsai March 8, 2011 at 6:26 am
for ((i=1; i<=20; i++)); do printf "%02d\n" "$i"; done

Reply

154 nixCraft March 8, 2011 at 6:37 am

+1 for printf due to portability, but you can use bashy .. syntax too

for i in {01..20}; do echo "$i"; done

Reply

155 TheBonsai March 8, 2011 at 6:48 am

Well, it isn’t portable per se, it makes it portable to pre-4 Bash versions.

I think a more or less “portable” (in terms of POSIX, at least) code would be

i=0
while [ "$((i >= 20))" -eq 0 ]; do
  printf "%02d\n" "$i"
  i=$((i+1))
done

Reply

156 ankit gulati March 12, 2011 at 2:44 pm

Hi

Can anybody help for this.

I have two text file viz gem1.txt and gem2.txt

EX. Content of gem1.txt

activerecord (2.3.5, 2.2.2)
activerecord-oracle_enhanced-adapter (1.1.9)
activerecord-sqlserver-adapter (2.3.4)
activeresource (2.3.5, 2.2.2)

Now i have to put all the data of these two .txt file on gem.csv. column wise.

Reply

157 Philippe Petrinko March 12, 2011 at 10:58 pm

@ankit gulati

This is not a place :-P
for such questions, go and see Forum. http://nixcraft.com/

Anyway an answer is [paste] command. :-)
http://linux.die.net/man/1/paste

Reply

158 PiCarre March 20, 2012 at 9:15 am

Simply use the “paste” command.

Reply

159 Rupender March 23, 2011 at 3:48 pm

Hi
can some one help in ceating a loop

the problem is
i have 3 folder and name of the folder changes.
i want a loop which enter these folder one by one echo the files inside the folder

Reply

160 Philippe Petrinko March 23, 2011 at 7:27 pm

Hi

following this topic,
you may try to write something down first,
and then we may help you … if you help yourself first.

Read and use this topic,
and the following material: http://bash.cyberciti.biz/guide/Main_Page

However, such request would be best posted into the forum http://nixcraft.com/

Anyway, a simple solution would be to use [ls] command with appropriate arguments.

Reply

161 Philip Ratzsch April 20, 2011 at 5:53 am

I didn’t see this in the article or any of the comments so I thought I’d share. While this is a contrived example, I find that nesting two groups can help squeeze a two-liner (once for each range) into a one-liner:

for num in {{1..10},{15..20}};do echo $num;done

Great reference article!

Reply

162 Philippe Petrinko April 20, 2011 at 8:23 am

@Philip
Nice thing to think of, using brace nesting, thanks for sharing.

Reply

163 David May 4, 2011 at 10:10 am

Hi guys.
I was wondering whether it is possible to run a for loop through rsh (now before you start protesting that I use rsh instead of ssh; I know that it is extremely insecure. Thanks.) like this:?
rsh -l dev 192.168.x.x “for file in /PackSun/distills/*; do if [[ ${file} =~ "3\.6\.76" ]]; echo $file; fi; done”

Because when I type this it doesn’t work, $file is always the same file and it comes from the pwd on the local machine, not on the remote.
Thanks in advance.
D

Reply

164 David May 4, 2011 at 10:11 am

damn, sorry, didn’t close the code tag:

rsh -l dev 192.168.x.x “for file in /PackSun/distills/*; do if [[ ${file} =~ "3\.6\.76" ]]; echo $file; fi; done”

Reply

165 Anthony Thyssen May 5, 2011 at 2:57 am

First — you should not use rsh. It is an old and insecure network protocol. ssh is its replacement.

As for your question YES it is posible, Wrap it in a ‘shell’ command.

rsh -l dev 192.168.x.x ‘sh -c ‘\”for file in /PackSun/distills/*; do if [[ ${file} =~ "3\.6\.76" ]]; echo $file; fi; done’\’

Watch the quoting as you want to use single quotes for both the outside and inside commands. If you use ” quotes you will need to escape the $ characters instead! And that may in turn lead to escapes of escapes (not pretty).

I have actually done this to execute a very very large shell script on a remote server.
However I don’t recommend it for things beyond what you have. If need be copy (update) a shell script on the remote server (scp, rsync, etc) then execute that. It works better, you do not have constant quote handling problems, and not command line length limits.

I do this for a incremental home backup program (written for my own use, using rsync hardlinks backup directories), that can backup to a remote account. You are free to download and look at the copy/execute remote script that the main script performs for its sub-scripts (like backup cycle rolls).

Reply

166 Philippe Petrinko May 5, 2011 at 1:10 pm

Anthony,

You forgot to include URI for you script, your > a < tag is empty.

Reply

167 Anthony Thyssen May 5, 2011 at 3:01 am

PS: the reason for the ‘sh -c …’ is beause you may nto have control of the login shell of the remote account. If you did not have it and the remote accoutn used tcsh or zsh, you will have problems.

PPS; you are missing a then. I tested it with this ssh version (a shell-builtin only ‘ls’ of the remote account).
ssh remote_machine ‘sh -c ‘\”for file in *; do echo $file; done’\’

Reply

168 Sanya May 6, 2011 at 6:02 am

Hello

I am using bash V4+ and this loop works fine:

for i in {1..10}; do echo $i; done

But when I put variables in the loop

max=10; for i in {1..$max}; do echo $i; done

I see one line as output:

{1..10}

Could anybody explain me what’s wrong ?

Cheers. Sanya

Reply

169 Philippe Petrinko May 6, 2011 at 10:13 am

Hello Sanya,

That would be because brace expansion does not support variables. I have to check this.
Anyway, Keep It Short and Simple: (KISS) here is a simple solution I already gave above:

xstart=1;xend=10;xstep=1
for (( x = $xstart; x <= $xend; x += $xstep)); do echo $x;done

Actually, POSIX compliance allows to forget $ in for quotes, as said before, you could also write:

xstart=1;xend=10;xstep=1
for (( x = xstart; x <= xend; x += xstep)); do echo $x;done

Reply

170 Philippe Petrinko May 6, 2011 at 10:48 am

Sanya,

Actually brace expansion happens __before__ $ parameter exapansion, so you cannot use it this way.

Nevertheless, you could overcome this this way:

max=10; for i in $(eval echo {1..$max}); do echo $i; done

Reply

171 Sanya May 6, 2011 at 11:42 am

Hello, Philippe

Thanks for your suggestions
You basically confirmed my findings, that bash constructions are not as simple as zsh ones.
But since I don’t care about POSIX compliance, and want to keep my scripts “readable” for less experienced people, I would prefer to stick to zsh where my simple for-loop works

Cheers, Sanya

Reply

172 Philippe Petrinko May 6, 2011 at 12:07 pm

Sanya,

First, you got it wrong: solutions I gave are not related to POSIX, I just pointed out that POSIX allows not to use $ in for (( )), which is just a little bit more readable – sort of.

Second, why do you see this less readable than your [zsh] [for loop]?

for (( x = start; x <= end; x += step)) do
echo "Loop number ${x}"
done

It is clear that it is a loop, loop increments and limits are clear.

IMNSHO, if anyone cannot read this right, he should not be allowed to code. :-D

BFN

Reply

173 Anthony Thyssen May 8, 2011 at 11:30 pm

If you are going to do… $(eval echo {1..$max});
You may as well use “seq” or one of the many other forms.
See all the other comments on doing for loops.

Reply

174 Tom P May 19, 2011 at 12:16 pm

I am trying to use the variable I set in the for line on to set another variable with a different extension. Couldn’t get this to work and couldnt find it anywhere on the web… Can someone help.

Example:

FILE_TOKEN=`cat /tmp/All_Tokens.txt`
for token in $FILE_TOKEN
do
A1_$token=`grep $A1_token /file/path/file.txt | cut -d ":" -f2`

my goal is to take the values from the ALL Tokens file and set a new variable with A1_ infront of it… This tells be that A1_ is not a command…

Edited by admin: Added pre taga

Reply

175 atef fawzy May 23, 2011 at 10:27 am

could you please help me in write a bash script do the following:
1- read an input file contains IP Port
2- check the connectivity for every IP and its Port via telnet
3- if the telnet not connected send email to alert me.

thanks in advance
atef fawzy

Reply

176 Philippe Petrinko May 23, 2011 at 11:34 am

No problem.
Would you first send me 1000USD payment?
Thanks in advance. ;-)

Reply

177 atef fawzy May 23, 2011 at 12:37 pm

dear Philippe Petrinko
thank you a lot for you value site
the below is my script and i don’t know what is the wrong?can you help me

#!/bin/sh
for ip in $(cat iplist);

do
# check for open ports #

connTest=`echo ” ” | telnet $ip`

if [ "`echo $ip | awk '{ print $3 }` = "Connected" ]
then
echo “$ip is up”

else

#echo “$connTest port is down”
echo $ip is down

fi
done

Reply

178 Philippe Petrinko May 23, 2011 at 12:25 pm

Lucky You! Atef fawzy, Vivek has already written necessary training material for you: Read http://bash.cyberciti.biz/guide/Main_Page

Reply

179 atef fawzy May 23, 2011 at 12:41 pm

please tell me what is the wrong?

#!/bin/sh
for ip in $(cat iplist);

do
# check for open ports #

connTest=`echo ” ” | telnet $ip`

if [ "`echo $ip | awk '{ print $3 }` = "Connected" ]
then

echo “$ip is up”

else

echo $ip is down

fi
done

Reply

180 atef fawzy May 23, 2011 at 12:33 pm

dear Philippe Petrinko
thank you a lot for your value site
the below is my script and i don’t know what is the wrong? can you hep me please

#!/bin/sh
for ip in $(cat iplist);

do
# check for open ports #

connTest=`echo ” ” | telnet $ip`

#if [ "`echo $connTest | awk '{ print $3 }` " = "Connected" ]
if [ "`echo $ip | awk '{ print $3 }` = "Connected" ]
then

#echo “$connTest port is up”
echo “$ip is up”

else

#echo “$connTest port is down”
echo “$ip is down”

fi
done

Reply

181 Philippe Petrinko May 23, 2011 at 1:39 pm

Your request is off-topic.
Your [for] loop works fine, so this is not a point to be discussed here,

Your [if] test is buggy – not to be discussed here but there http://nixcraft.com/

Reply

182 atef fawzy May 24, 2011 at 10:20 am

please send me the full URL to post the error i got

Reply

183 Philippe Petrinko May 24, 2011 at 11:14 am

You can go where I already offered you => http://nixcraft.com/

You can go there and figure out what topic is appropriate to your needs.

Reply

184 loop how May 26, 2011 at 12:49 am

#!/bin/sh
How can I do this using infinite loops?

for disk in $(ls /dev/disk/by-id/ | grep -v part | grep scsi-35000I);
do
echo $disk && \
dd if=/dev/zero of=/dev/disk/by-id/$disk bs=8M oflag=direct count=250 && \
dd if=/dev/disk/by-id/$disk of=/dev/null bs=8M count=250;
done

Reply

185 Chu-Siang July 21, 2011 at 9:07 pm

thanks for you example, It`s help me very much.

Reply

186 Faizanul Islam July 26, 2011 at 6:04 am

hey,

Thanks a lot. Example are in the way they should be…some of the examples are very new to me. It has increased by knowledge.
keep posting these things and let us increase our knowledge.

Reply

187 brian August 5, 2011 at 7:58 am

Hi,

I have to read a file line by line and do the iteration and apply the condition parameter for each input . Please share your ideas

Reply

188 Philippe Petrinko August 5, 2011 at 9:56 am

@Brian
You could google “awk” to proceed a file line by line, of use example in Vivek’s blog.

http://bash.cyberciti.biz/file-management/read-a-file-line-by-line/
http://www.cyberciti.biz/faq/unix-howto-read-line-by-line-from-file/

Reply

189 Sam McAllister August 10, 2011 at 11:10 am

Hi, I’m a beginner and I was writing a very simple script :

#!/bin/sh
clear
echo “Enter username:”
read username
if [ "$username" = "newbay" ]
then
echo “Username correct,”
else
echo “Incorrect username, try again:”

fi

echo “Now enter password:”
read password
if [ "$password" = "welcome" ]
then
echo “You are now logged in.”
else
echo “Sorry, incorrect password. Please try again.”

fi
and I was wondering how to loop the incorrect username try again part ? Can anyone help me, if they understand my awful script ..

Reply

190 Anthony Thyssen August 11, 2011 at 7:25 am

Start a new thread!

Reply

191 Sam McAllister August 11, 2011 at 7:47 am

Haha ok thanks :)

Reply

192 Philippe Petrinko August 11, 2011 at 8:48 am

You could start learning shell scripting:

About [for] loops: http://bash.cyberciti.biz/guide/For_loop

( from that excellent http://bash.cyberciti.biz/guide/Main_Page )

Enjoy!

Reply

193 ITtuition.com August 11, 2011 at 1:30 pm

Spot on. Included all examples that form a good base. Thanks for sharing.

Reply

194 Nagesh September 14, 2011 at 10:21 am

This post is 3 yrs old but still RockS.\m/

Reply

195 ptrac3 October 24, 2011 at 5:43 pm

Very handful tutorial! : ) But i’ve got a problem on my Ubuntu linux box, a strange problem.. Executing the example code

#!/bin/bash
for i in {1..5}
do
echo “Welcome $i times”
done

i get as output Welcome {1..5} times. So the bash doesn’t understand the range {1..5}…Have any idea? My bash version is GNU bash, version 4.2.8(1)-release (x86_64-pc-linux-gnu). Thank u : )

Reply

196 Philippe Petrinko October 25, 2011 at 9:01 am

This code has to be written into a text file, which must be made executable.
Did you do that?

Reply

197 nixCraft October 25, 2011 at 11:19 am

GNU/Bash v4.2.8 does supports {1..5} syntax. What is the output of the following commands?

ls -l /bin/bash
/bin/bash --version

Reply

198 Sonia McMath November 18, 2011 at 6:52 am

Great weblog right here! Additionally your web site quite a bit up very fast! What host are you the usage of? Can I get your associate hyperlink for your host? I want my website loaded up as fast as yours lol

Reply

199 Hannah December 5, 2011 at 11:21 pm

Hi all… I have a question about using for loops. Basically, I have a file containing a list of protein ID numbers. what I want to do is create a for loop that can go into this file, grab each protein ID number, and then search the NCBI database to get the fasta sequence of each protein… and then create another file containing all of the fasta sequences….

In general, my problem is that I can’t figure out how to get the protein ID numbers from the output file (ex. $1 in file1) into the for loop script.

This is what I have so far:

for gi in file1
do
fastacmd -d /data/nr -s gi
done

but I need to specify that the gi (protein ID number) is the first column ($1) of file1.

Does this make sense?

Reply

200 Akshit December 7, 2011 at 12:36 pm

Hi…
Thanks for the article, it is helpful really.

I want to know one thing.
Let’s say there is file with content
1
2
3
4
5

how can I use for loop and print these numbers?
If you are understanding what I am trying to tell. I want to use the contents of these files and stored in the variable.

Reply

201 andreea January 5, 2012 at 4:21 pm

Hello!
i am beginner and i have to make a bash script in which i have to show the number of lines that were added by the analized commits(in git).can you help me?Thanks.

Reply

202 Philippe Petrinko January 5, 2012 at 5:33 pm

Dear andreea

1) I quote Vivek: “You are free to use our shell scripting forum for questions.”
http://nixcraft.com/shell-scripting/

2) You can learn for free shell scripting with the infamous Vivek’s collaborative Linux Shell Scripting Tutorial
http://bash.cyberciti.biz/guide/Main_Page

Enjoy!

– Philippe

Reply

203 Philippe Petrinko January 5, 2012 at 6:23 pm

“infamous” was a private joke-understatement
Sorry, no offense intended!
Vivek website roxxxxs !

– Philippe

Reply

204 andreea January 6, 2012 at 6:33 pm

Thanks for your quick answer.
I have another problem.i have to use this script by using git.can you explain me or give some advices how can i use it?thank you very much.

Reply

205 andreea January 8, 2012 at 8:54 pm

I have this code that shows the first column which represents the number of insertions of a commit and calculate the sum of all numbers of each line.
if [ $# -eq 2 ]; then
if [ $2 = "added_lines" ]; then
tmpfile=$(mktemp)
sum=0
git log –pretty=tformat: –numstat | tr -s “\n” | cut -f 1 > $tmpfile
for i in $(cat $tmpfile); do
sum=$(($sum + $i))
done
echo “$sum”
rm $tmpfile
But with ‘cut -f 1′ it takes too much time to calculate and to show the result.Can you help me saying how can i do this with ‘awk’ or ‘gawk’?
Please help me.Thanks

Reply

206 Ram January 10, 2012 at 6:57 pm

Hi Vivek,
i am trying to run the below code as ram.sh in server as $ sh ./ram.sh

code:

————————-
#!/bin/bash
for i in 1 2 4
do
echo “Welcome $i times”
done
————————-

It fails saying
ram.sh: line 3: syntax error near unexpected token `do
ram.sh: line 3: `do

is that something i need to check which version is the sh and bash used in the server. how to check that.
it is a pretty simple code and it is not workign.
i am running this script in Linux server 64 bit server.

Reply

207 the dsc October 23, 2012 at 2:54 am

What’s the easiest way to have a loop from N to N, when the numbers are dictated by variables?

Something like what one could/would expect from “for i in {$a..$b}”, but something that actually works of course.

What I do is:

a=0 ; b=5 ; until ((a==b)) ; do echo $a ; a=$(($a+1)) ; done

But it seems kind of dumb, somewhat like echoing a huge set of spaces instead of “clear”, to clear the screen. But perhaps it’s the only way to do it. And it works anyway, so perhaps it’s good enough and worthy as an addition rather than a question.

Funnily enough you can make an alphabetic countdown with “for i in {a..z}”

Reply

208 Philippe Petrinko January 27, 2012 at 10:07 am

Hi Ram,

Since you are a real beginner, you would really take great benefit self-training on this course free of charge by vivek & contributors:
http://bash.cyberciti.biz/guide/Main_Page

–P

Reply

209 shahzad March 10, 2012 at 9:04 am

Hi,
I have this code in NetBeens:
#include
#include
using namespace std;
int main() {
int a,i,j;
cout << "inter your number of row " <>a;
for (i=0; i<=a; i++)
{
for(j=0; j<=i; j++)
{
cout << "*";
}
cout << '\n';

}
return 0;
}

but I want to run it in ubuntu so at first I made a " nano file.sh" aftaer that I wrote this code in it:
#!/bin/sh
echo “Enter your number of rows”
Read a
for((i=0; i<=a; i++))
do
for((j=0; j<=i; j++))
do
echo “*”
done
echo –n
done
exit 0

after that " chmod +x file.sh", but my code didn't run, my code in ubuntu has problem,
I don't know how can I solve it, Thanks a lot if you answer my question,
Regards

Reply

210 regan May 5, 2012 at 2:26 pm

hi, am having problem to write a program using For statement. Please help if you know something about using For statement

Reply

211 Excellent_Test August 23, 2012 at 11:57 am

This is what I was looking for. TLDP advanced guide is also good one.

Reply

212 Philippe Petrinko November 12, 2012 at 7:38 pm

Vivek,
It seems that Wordpress was hungry, it has eaten your text ;-)

[for] code sample is broken after sentence: “A representative three-expression example in bash as follows”

–P

Reply

213 Youshi Patel February 20, 2013 at 3:14 pm

nice write up.

Reply

214 m.vinothini March 7, 2013 at 6:17 am

very useful

Reply

215 OSuKaRu June 6, 2013 at 8:54 pm

i try the infinite variable, but it tells me that:

Syntax error: Bad for loop variable

:<

Reply

216 K J Ramana Rao June 20, 2013 at 2:26 pm

Hello,
Nice post.

Please help me in below code. we are getting syntax error. Please resolve the syntax issue.

if [ $1 ] ; then
    LIB=$1/lib
    if -d ${_LIB} ; then
        for jar in (cd ${_LIB}; ls *.jar)
        do
            EXISTS=echo ${CP} | grep "/${jar}"
            if [ "${_EXISTS}" != "" ]; then
                logger "WARN: Classpath will contain multiple files named ${jar}"
            else
                CP=${CP}${CP_DELIM}${LIB}/${jar}
            fi
        done
    fi
    cd $1
fi
}

Error:

syntax error near unexpected token `('
line 15: `for jar in (cd ${_LIB}; ls *.jar)'

Thanks
Ramana

Reply

217 Anthony Thyssen June 24, 2013 at 12:43 am

You are a ‘help troll’.. You posted in the wrong section.
you would have been better of starting a new topic.

Reply

218 foolar June 21, 2013 at 1:20 pm

try with

if [ $1 ] ; then
LIB=${1}/lib
if -d ${_LIB} ; then
for jar in $(cd ${_LIB}; ls *.jar)
do
EXISTS=$(echo ${CP} | grep “/${jar}”)
if [ "${_EXISTS}" != "" ]; then
logger “WARN: Classpath will contain multiple files named ${jar}”
else
CP=${CP}${CP_DELIM}${LIB}/${jar}
fi
done
fi
cd $1
fi

Reply

219 abr June 27, 2013 at 12:51 am

hi if I use it with a imput variable like:

echo “enter the numbers of repetitions :”
read variable

tthe variable represent the number example 15 times to repeat !!!???/

how could it be ???

Reply

220 dennis bernaerts July 4, 2013 at 1:19 pm

nice and clear thx a lot.
What I am dreaming of is this clear kind of webpage in which each box has a tab in which you can select any language…. I’m sure it exists already but where ???

and I mean it needs to be clear and simple !!

Reply

221 Tring Tring September 24, 2013 at 9:27 am

Thank you so Much!

Reply

222 sana October 2, 2013 at 10:40 am

hi i have a problem i want to write a shell script for siesta.
i want to make diffrent directories and want to change lattice constants n then run it with siesta n want to chck total energy
cn u tell m how cn i do this

Reply

223 Philippe Petrinko October 2, 2013 at 12:32 pm

no w cnt
sry w dnt 1drstnd w U sd

Reply

224 sana October 3, 2013 at 6:26 am

I want to write a shell script
for diffrent values of lattice constants .
If i do manually i first make a directorie i.e for 3.80 then i copy *.psf and fdf file there open the fdf file and change lattice constant to 3.80 and the execute file with siesta and note the total energy.
n again i repeat the same process for let say 3.90,4.10.4.20 etc.
now i want to write a code in shell so that i dnt need to make directory every time and change lattice constant.
I want to use looping for this purpose……..but how??
anybody have the little code for this?or anything?

Reply

225 sana October 3, 2013 at 6:31 am

#!/bin/bash
# changing lattice constants
for 3.80 3.90 in $inp3.70.fdf
do
echo 3.80 3.90
mkdir
mv inp3.70.fdf inp*.fdf.old
mv *.psf *.psf.old
sed ‘s/inp*.fdf/&, lattice constant/’ inp*.fdf
done
# Now comes the commands to be executed
~/code/sanabin/siesta 3.70.out &
~
as i write this but not working…….. ../

Reply

226 Philippe Petrinko October 3, 2013 at 7:48 am

Well, it seems that you do not understand the basics of shell scripting, even a simple [mkdir] command.

Anyway, you are very lucky, because this web site and Internet offer you _free_ training material.

The good news is : all you have to do is learn and work, using for instance these links.

http://bash.cyberciti.biz/guide/Main_Page
http://bash.cyberciti.biz/script/for-loop/
and of course this page http://www.cyberciti.biz/faq/bash-for-loop/

Reply

227 matthias October 10, 2013 at 5:19 am

nice you used “c++” in “A representative three-expression example in bash” but that is a programing language.

Reply

228 Philippe Petrinko October 10, 2013 at 9:18 am

@matthias :

What’s your point?

Is it a humorous play of words on “C++” language and increment of c variable in this script?

In that case it would have been more convenient to append a smiling smiley, because otherwise, there is no coding issue in “A representative three-expression example in bash”.

:-/ wondering…

– Philippe

Reply

229 ravindra December 3, 2013 at 3:00 pm

Hi.. check it

for (( c=1; c<=5; c++ ))
do
   echo "Welcome $c times"
done

why came this error in ubuntu please tell me
reply please
./for3: 1: ./for3: Syntax error: Bad for loop variable

Reply

230 RH User January 3, 2014 at 9:12 pm

Thanks. Solved my coding issue.

Reply

231 dee January 17, 2014 at 8:27 pm

Can anyone advise how to write a for loop statement to run commands from a .sh when an event comes up like users uploading new files to the server.

My issue is that I am using a soft link to mirror an external disk drive in the .www/ and the soft link never updates when a new content is added to the drive within a session.

Reply

Leave a Comment

Tagged as: , , , , , , , , , ,

Previous Faq:

Next Faq: