Q. I want to change the formatting of a file. I just wanted to remove all blank lines from text file. How do I achieve this task w/o spending much time?
A. Yes, you do not have to waste your time making manual changes to files. Both Linux and UNIX systems come with file manipulation tools that can be used to remove all blank lines very quickly.
Task: Remove blank lines using sed
Type the following command:
$ sed '/^$/d' input.txt > output.txt
Task: Remove blank lines using grep
$ grep -v '^$' input.txt > output.txt
Both grep and sed use special pattern ^$ that matchs the blank lines. Grep -v option means print all lines except blank line.
Let us say directory /home/me/data/*.txt has all text file. Use following for loop (shell script) to remove all blank lines from all files stored in /home/me/data directory:
#!/bin/sh files="/home/me/data/*.txt" for i in $files do sed '/^$/d' $i > $i.out mv $i.out $i done
Updated for accuracy.
Featured Articles:
- 20 Linux System Monitoring Tools Every SysAdmin Should Know
- My 10 UNIX Command Line Mistakes
- 10 Greatest Open Source Software Of 2009
- Top 5 Email Client For Linux, Mac OS X, and Windows Users
- Top 20 OpenSSH Server Best Security Practices
- Top 10 Open Source Web-Based Project Management Software
- Top 5 Linux Video Editor Software
- Email FAQ to a friend
- Download PDF version
- Printable version
- Comment RSS feed
- Last Updated: 08/10/08
{ 12 comments… read them below or add one }
Is that last shell script wrong? The for loop is using variable f, and inside the loop, everything is using variable i. I think the script should look more like:
#!/bin/sh
files=”/home/me/data/*.txt”
# Next line is changed to use variable i, not f
for i in $files
do
sed ‘/^$/d’ $i > $i.out
mv $i.out $i
done
its nice..appreciable
James,
Thanks for the heads up. The faq has been updated.
When writing to the same file, or a different file and using the redirect > the file ends up blank.
This removes blank lines from input.txt
sed ‘/^$/d’ -i input.txt
Hi,
I tried to use the command sed given above to remove blank line from my file.
The file looks good except one strange behaviour:
“Always” the last line of the file is removed..for example:
test.p is the file and following are the contents of the file:
BEFORE RUNNING THE SED COMMAND:
“this is test 1.
this is test 2.
this is test 3.”
AFTER RUNNING THE SED COMMAND:
“this is test 1.
this is test 2.”
As you can see the last line (“this is test 3.”) is removed after running the command.
If I put the blank line at the end it removes the blank line i.e.
“this is test 1.
this is test 2.
this is test 3.
”
becomes…….
“this is test 1.
this is test 2.
this is test 3.”
Do let me know what I should do in this case?
Waiting for your prompt reply…!
Thanks and Best Regards,
Prashant Deshani
Actually, grep’s -v (–invert-match) option inverts the sense of matching, thus selecting non-matching lines.
Therefore, only when combining it with the pattern ‘^$’ does it mean to select everything except blank lines.
Just thought the way you explained it might be confusing for beginners. When I was new to shell and Linux, most of the guides or forums were useless because they all assumed that the person that needs help has some high understanding of shell or Linux already, and thus they use a lot of terms that person would have never heard of before or they don’t explain anything. Once you have a basic understanding, the learning comes easier, but in the beginning, the learning curve has an almost horizontal slope.
Hi,
I am Siva Saran
I am new to linux environment
can any on tell plz that how to get a particular line from a file using linux commands
For example say that inputfile has lines as follows
total 0
d [RWCEAFMS] MRohit 512 Apr 15 03:15 3.1_SP1_Beta1_127
d [RWCEAFMS] MRohit 512 Apr 14 03:26 3.1_SP1_125 d [RWCEAFMS] MRohit 512 Apr 14 03:26 3.1_SP1_259
d [RWCEAFMS] MRohit 512 Apr 14 03:26 3.0_SP4_IR3_62
Now I want the line which is having SP1_125 ie., line number 3 and store it in an variable
plz can any tell the answer plz
can u mail to {snip_no_email_ids}
what if there’s spaces and tab in the line I want to delete?
thanks,
Renz
Hi,
Using the command
grep -R “SP1_125″
you can get the lines with SP1_125.
if you want to assign to a variable then use
var_a=$(grep -R “SP1_125″ )
where is the name of file
var_a = $(grep -R “SP1_125″ <filename>)
Hi,
var_a=$(grep -R “SP1_125″ filename)
This is quite helpful. Thanks!!