sed tip: Remove / Delete All Leading Blank Spaces / Tabs ( whitespace ) From Each Line

last updated in Categories Howto, Linux, Shell scripting, UNIX

The sed (Stream Editor) is very powerful tool. Each line of input is copied into a pattern space. You can run editing commands on each input line to delete or change the input. For example, delete lines containing word DVD, enter:
cat input.txt | sed ‘/DVD/d’

To Print the lines between each pair of words pen and pencil, inclusive, enter:
$ cat input.txt sed -e '/^PEN/,/^PENCIL/p'

To remove all blank lines, enter:
$ cat /etc/rssh.conf | sed '/^$/d' > /tmp/output.file

sed is very handy tool for editing and deleting unwanted stuff. Following echo statements printed lots of whitespace from left side:
echo "     This is a test"
Output:

         This is a test

To remove all whitespace (including tabs) from left to first word, enter:
echo "     This is a test" | sed -e 's/^[ \t]*//'
Output:

This is a test

Where,

  • s/ : Substitute command ~ replacement for pattern (^[ \t]*) on each addressed line
  • ^[ \t]* : Search pattern ( ^ – start of the line; [ \t]* match one or more blank spaces including tab)
  • // : Replace (delete) all matched pattern

Following sample script reads some data from text file and generate a formatted output. It delete all leading whitespace from front of each line so that text get aligned to left:

#!/bin/bash
FILE=url.dump.txt
DOMAIN=yourdomain.com
exec 3<&0
exec 0<$FILE
while read line
do
	url=$(echo "http://${DOMAIN}${line}")
        title="$(lynx -dump -source ${url} | grep '<title>' | awk -F '<title>' '{ print $2 }' | cut -d'<' -f1|sed 's/^[ \t]*//')"
        echo "<li>${title}</li>"
done
exec 0<&3

To delete trailing whitespace from end of each line, enter:
$ cat input.txt | sed 's/[ \t]*$//' > output.txt
Better remove all leading and trailing whitespace from end of each line:
$ cat input.txt | sed 's/^[ \t]*//;s/[ \t]*$//' > output.txt

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

14 comment

  5. Some nice examples, but there’s unnecessary use of a cat process for the examples with input.txt. For example,
    cat input.txt | sed ‘/DVD/d’ is more efficient as sed ‘/DVD/d’ input.txt
    $ cat input.txt sed -e ‘/^PEN/,/^PENCIL/p’ is more efficent as sed -e ‘/^PEN/,/^PENCIL/p’ input.txt

  8. better example yet….

    cat /usr/bin/scrub
    # Remove all comments and empty lines
    sed -e ‘s/#.*//’ -e ‘s/[ ^I]*$//’ -e ‘/^$/ d’ $1

    cat /usr/bin/clrspace
    # Remove all leading and trailing whitespace from each line;
    sed ‘s/^[ t]*//;s/[ t]*$//’ $1

    now you can simply run
    scrub filename
    clrspace filename

  9. I tried your sed syntax for leading/trailing space/tabs removal and it doesn’t do it as expected:

    # sed s/$/G/g test_file
    /etcG
    /homeG
    /usr/sbin/snapshotsG
    /usr/local/etcG
    /var/spool/cron/crontabsG
    G
    / G
    / /dgdkjgd G
    /G
    / /fff/f/ff G
    G
    /sdfsfssf G
    #/sfsfsfG
    /sfsd/sfd / G
    /sfsd/sfdsfs /dfgf/df/G
    /dfgdg /gddd /G
    # sed ‘s/^[ t]*//;s/[ t]*$//’ test_file|sed s/$/G/g
    /etcG
    /homeG
    /usr/sbin/snapshotsG
    /usr/local/etcG
    /var/spool/cron/crontabsG
    G
    /G
    / /dgdkjgd G
    /G
    / /fff/f/ff G
    G
    /sdfsfssfG
    #/sfsfsfG
    /sfsd/sfd / G
    /sfsd/sfdsfs /dfgf/df/G
    /dfgdg /gddd /G
    #

  11. Hi,
    how can I remove spaces at the end of each line in a *.txt file…e.g.
    word1[space]
    word2[space]
    word3[space]
    .
    .
    word n[space]
    commands you stated didnt work

    Have a question? Post it on our forum!

Tagged as: Tags , , , , , , ,