31 comment

1. Is that last shell script wrong? The for loop is using variable f, and inside the loop, everything is using variable i. I think the script should look more like:

   #!/bin/sh
   files=”/home/me/data/*.txt”
   # Next line is changed to use variable i, not f
   for i in $files
   do
   sed ‘/^$/d’ $i > $i.out
   mv $i.out $i
   done

   1. Hello!

      Little fix in this BASH script:

      #!/bin/bash
      #!/bin/sh
      files=”/home/tiger/test_moduls/*.txt”
      for i in $files
      do
      sed ‘/^ *$/d’ $i > $i.out
      mv $i.out $i
      done

      Good Luck! :-)

2. its nice..appreciable

3. James,

   Thanks for the heads up. The faq has been updated.

4. When writing to the same file, or a different file and using the redirect > the file ends up blank.

   This removes blank lines from input.txt

   sed ‘/^$/d’ -i input.txt

5. Hi,

   I tried to use the command sed given above to remove blank line from my file.

   The file looks good except one strange behaviour:
   “Always” the last line of the file is removed..for example:
   test.p is the file and following are the contents of the file:

   BEFORE RUNNING THE SED COMMAND:
   “this is test 1.

   this is test 2.

   this is test 3.”

   AFTER RUNNING THE SED COMMAND:
   “this is test 1.
   this is test 2.”

   As you can see the last line (“this is test 3.”) is removed after running the command.

   If I put the blank line at the end it removes the blank line i.e.
   “this is test 1.

   this is test 2.

   this is test 3.

   ”

   becomes…….
   “this is test 1.
   this is test 2.
   this is test 3.”

   Do let me know what I should do in this case?

   Waiting for your prompt reply…!

   Thanks and Best Regards,
   Prashant Deshani

6. Actually, grep’s -v (–invert-match) option inverts the sense of matching, thus selecting non-matching lines.
   Therefore, only when combining it with the pattern ‘^$’ does it mean to select everything except blank lines.
   Just thought the way you explained it might be confusing for beginners. When I was new to shell and Linux, most of the guides or forums were useless because they all assumed that the person that needs help has some high understanding of shell or Linux already, and thus they use a lot of terms that person would have never heard of before or they don’t explain anything. Once you have a basic understanding, the learning comes easier, but in the beginning, the learning curve has an almost horizontal slope.

7. Hi,
   I am Siva Saran
   I am new to linux environment
   can any on tell plz that how to get a particular line from a file using linux commands
   For example say that inputfile has lines as follows
   total 0
   d [RWCEAFMS] MRohit 512 Apr 15 03:15 3.1_SP1_Beta1_127
   d [RWCEAFMS] MRohit 512 Apr 14 03:26 3.1_SP1_125 d [RWCEAFMS] MRohit 512 Apr 14 03:26 3.1_SP1_259
   d [RWCEAFMS] MRohit 512 Apr 14 03:26 3.0_SP4_IR3_62

   Now I want the line which is having SP1_125 ie., line number 3 and store it in an variable
   plz can any tell the answer plz
   can u mail to {snip_no_email_ids}

8. what if there’s spaces and tab in the line I want to delete?

   thanks,
   Renz

   1. This is old, but in case no-one told you:

      \w is the special regular expression for whitespace. Both the sed and grep matches should be changed to:

      /^\w*$/

      -Angwe

9. Hi,

   Using the command
   grep -R “SP1_125”
   you can get the lines with SP1_125.
   if you want to assign to a variable then use
   var_a=$(grep -R “SP1_125” )

   where is the name of file

10. var_a = $(grep -R “SP1_125” <filename>)

11. Hi,
    var_a=$(grep -R “SP1_125” filename)

12. This is quite helpful. Thanks!!

13. how count all lines in all files in current dir and omit empty lines with wc, grep, cut and bc commands

    echo `wc -l * | grep total | cut -f2 -d’ ‘` – `grep -in “^$” * | wc -l ` | bc

    1. @sobi3ch
       LOL, wut? Wanna win an obfuscation contest?

       grep -v '^\w*$' * | wc -l
       

       would be better, because it doesn’t count lines with only whitespaces, and uses only two instead of seven(!) programs.

       for f in *; do awk '!/^\w*$/ && !/^\w*#/ {x++} END {print FILENAME" :"x" lines"}' $f; done
       

       will print you the number of lines, excluding empty lines and lines starting with “#” for every file separately in the current directory.

    2. All you are trying to do is this:

       grep -v “^$” * | wc -l

14. The example should read

    sed -i '/^\w*$/d' input.txt
    

    grep -v '^\w*$' input.txt > output.txt && mv output.txt input.txt
    

    and

    #!/bin/sh
    files="/home/me/data/*.txt"
    for i in $files; do
        sed -i '/^\w*$/d' $i
    done
    

    Because
    a) using “^\w*$” instead of “^*$” removes also lines consisting only of whitespaces
    b) sed <em-i (=in-place) will edit the files directly, so that there is no need for output redirection and file renaming.

    If you have an non-GNU sed version, which doesn’t support in-place editing, than at least change the sed/mv lines to

    sed '/^\w*$/d' $i > $i.out && mv $i.out $i
    

    otherwise you could end up with a lot of empty files, if the sed command goes wrong.

    1. “sed <em-i" should read sed i

       1. “sed <em-i" should read "sed -i“

15. I would like to know how i can write a shell script to delete “datetime message” and a particular patern exists? in short I want my file to this following output.

    ### original file ###
    datetime message
    2011-07-13 13:45:35 Hello World of War Craft
    2011-07-13 13:48:43 This is a Test of text

    ### output file ###
    20110713134535 | Hello World of War Craft
    20110713134843 | This is a Test of text

    thanks

    1. Type

       echo '2011-07-13 13:45:35 Hello World of War Craft'  | sed -e 's/-//g' -e 's/://g' | awk '{ print $1$2 " | " substr($0, index($0,$3))}'

       Result:

       20110713134535 | Hello World of War Craft

       You need to process the file using while loop:

       while IFS= read -r line
       do
          echo "$line" | sed -e 's/-//g' -e 's/://g' | awk '{ print $1$2 " | " substr($0, index($0,$3))}'
       done < "input.txt"
       

       There may be a better solution to reduced awk, but I’m too lazy to try it out ;)

       1. Thanks for fast reply,

          when I run this shell script
          # while IFS= read -r line; do echo “$line” | sed -e ‘s/-//g’ -e ‘s/://g’ | awk ‘{ print $1$2 ” | ” substr($0, index($0,$3))}’; done testdb1.pl;

          the file output is below: the line 1 not remove and added datetimemessage, is there any way to remove the line 1 which is “datetimemessage | datetime message”

          datetimemessage | datetime message
          20110713134535 | Hello World of War Craft
          20110713134843 | This is a Test of text

16. while IFS= read -r line; do echo “$line” | sed -e ‘s/-//g’ -e ‘s/://g’ | awk ‘{ print $1$2 ” | ” substr($0, index($0,$3))}’; done testdb1.pl;

    1. correction
       ‘; done testdb.pl “>testdb.pl”

17. Hi all,

    I don’t if is the right to post this issue,
    is there any way to dump the new insert data to a textfile with shell script, for ie. after insertion to the mysql database execute the shell script to append and append the new record?

    thanks

18. Hi all,

    Is there a way to add an empty line for every 5 lines of data using grep?

    line1
    line2
    line3
    line4
    line5
    (empty line)
    line6
    line7
    line8
    line9
    line10
    (empty line)

19. you can also do cat output.txt | awk ‘NF’ >> output.txt

20. grep -v “^[[:blank:]]*$”

21. i have a text file that in end of file have a blank line but i cant delet in by sed i have to vi it and use dd command to delet it

22. I am using the command :-

    cat db_space.txt|sed '/-------------------- -------------------- --------------------/d'|sed '/^$/d'|sed '/^$/n'>db_space.txt

    I get the output like this,
    Database Size Used space Free space 22 GB 17 GB 5 GB in the output of text file.

    I want the output like this in the text file.

    Database Size	     Used space 	  Free space
    22 GB		     17 GB		  5 GB>

    Please let me know which command need to use?

Comments are closed.