sed tip: Remove / Delete All Leading Blank Spaces / Tabs ( whitespace ) From Each Line

Posted on in Categories Howto, Linux, Shell scripting, UNIX last updated December 30, 2007

The sed (Stream Editor) is very powerful tool. Each line of input is copied into a pattern space. You can run editing commands on each input line to delete or change the input. For example, delete lines containing word DVD, enter:
cat input.txt | sed ‘/DVD/d’

To Print the lines between each pair of words pen and pencil, inclusive, enter:
$ cat input.txt sed -e '/^PEN/,/^PENCIL/p'

To remove all blank lines, enter:
$ cat /etc/rssh.conf | sed '/^$/d' > /tmp/output.file

sed is very handy tool for editing and deleting unwanted stuff. Following echo statements printed lots of whitespace from left side:
echo "     This is a test"
Output:

         This is a test

To remove all whitespace (including tabs) from left to first word, enter:
echo "     This is a test" | sed -e 's/^[ \t]*//'
Output:

This is a test

Where,

  • s/ : Substitute command ~ replacement for pattern (^[ \t]*) on each addressed line
  • ^[ \t]* : Search pattern ( ^ – start of the line; [ \t]* match one or more blank spaces including tab)
  • // : Replace (delete) all matched pattern

Following sample script reads some data from text file and generate a formatted output. It delete all leading whitespace from front of each line so that text get aligned to left:

#!/bin/bash
FILE=url.dump.txt
DOMAIN=yourdomain.com
exec 3<&0
exec 0<$FILE
while read line
do
	url=$(echo "http://${DOMAIN}${line}")
        title="$(lynx -dump -source ${url} | grep '<title>' | awk -F '<title>' '{ print $2 }' | cut -d'<' -f1|sed 's/^[ \t]*//')"
        echo "<li>${title}</li>"
done
exec 0<&3

To delete trailing whitespace from end of each line, enter:
$ cat input.txt | sed 's/[ \t]*$//' > output.txt
Better remove all leading and trailing whitespace from end of each line:
$ cat input.txt | sed 's/^[ \t]*//;s/[ \t]*$//' > output.txt

14 comment

  1. Some nice examples, but there’s unnecessary use of a cat process for the examples with input.txt. For example,
    cat input.txt | sed ‘/DVD/d’ is more efficient as sed ‘/DVD/d’ input.txt
    $ cat input.txt sed -e ‘/^PEN/,/^PENCIL/p’ is more efficent as sed -e ‘/^PEN/,/^PENCIL/p’ input.txt

  2. better example yet….

    cat /usr/bin/scrub
    # Remove all comments and empty lines
    sed -e ‘s/#.*//’ -e ‘s/[ ^I]*$//’ -e ‘/^$/ d’ $1

    cat /usr/bin/clrspace
    # Remove all leading and trailing whitespace from each line;
    sed ‘s/^[ t]*//;s/[ t]*$//’ $1

    now you can simply run
    scrub filename
    clrspace filename

  3. I tried your sed syntax for leading/trailing space/tabs removal and it doesn’t do it as expected:

    # sed s/$/G/g test_file
    /etcG
    /homeG
    /usr/sbin/snapshotsG
    /usr/local/etcG
    /var/spool/cron/crontabsG
    G
    / G
    / /dgdkjgd G
    /G
    / /fff/f/ff G
    G
    /sdfsfssf G
    #/sfsfsfG
    /sfsd/sfd / G
    /sfsd/sfdsfs /dfgf/df/G
    /dfgdg /gddd /G
    # sed ‘s/^[ t]*//;s/[ t]*$//’ test_file|sed s/$/G/g
    /etcG
    /homeG
    /usr/sbin/snapshotsG
    /usr/local/etcG
    /var/spool/cron/crontabsG
    G
    /G
    / /dgdkjgd G
    /G
    / /fff/f/ff G
    G
    /sdfsfssfG
    #/sfsfsfG
    /sfsd/sfd / G
    /sfsd/sfdsfs /dfgf/df/G
    /dfgdg /gddd /G
    #

  4. Hi,
    how can I remove spaces at the end of each line in a *.txt file…e.g.
    word1[space]
    word2[space]
    word3[space]
    .
    .
    word n[space]
    commands you stated didnt work

Leave a Comment