Truncate Large Text File in UNIX / Linux

by Vivek Gite on February 20, 2010 · 11 comments

How do I truncate or shrink large text file under UNIX / Linux operating systems?

There are various tools to truncate large text files under UNIX / Linux operating systems.

Options #1: Shell Output Redirction

Your shell can truncate text file and make the size to zero using redirection:

> {filename}
ls -l largefile.txt
> largefile.txt
ls -l largefile.txt

Please note that largefile.txt file is created if it doesn't exist. And largefile.txt file is overwritten if it exits.

Option #2: truncate Command

Use the truncate command to shrink or extend the size of each FILE to the specified size:

truncate -s 0 {filename.txt}
ls -lh filename.txt
truncate -s 0 filename.txt
ls -lh filename.txt

The -s option is used to set SIZE to zero. See truncate command man page for more details:
man truncate

Option #3: logrotate Utility

logrotate command is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large. See how to use logrotate command to rotates, compresses, and mails system logs stored in /var/log and other locations under UNIX / Linux oses.

Featured Articles:

Share this with other sys admins!
Facebook it - Tweet it - Print it -

{ 11 comments… read them below or add one }

1 Nilesh February 20, 2010: Isn’t this more of a useless thing ? You lose all the data in the first two methods.
Reply
2 Vivek Gite February 20, 2010: @Nilesh,
That is the point, you want to truncate file size. For example, some stupid app crate a log file due to some networking issue. Next you fixed the issue and you don’t want the data. So just truncate it.
Reply
3 jaysunn February 20, 2010: I vote for Option#3. Can’t live without my logrotate.conf.
Jaysunn
Reply
4 Philippe Petrinko February 21, 2010: Option #4:
cp /dev/null toobig
Option #5:cat /dev/null > toobig
Reply
5 Philippe Petrinko February 21, 2010: Option #4:
cp /dev/null toobig
Option #5:
cat /dev/null > toobig
Reply
6 tiptop February 21, 2010: What does it mean to “rotate” log files?
Reply
7 Philippe Petrinko February 21, 2010: @tip
http://linux.die.net/man/8/logrotate
Reply
8 Shoaibi February 23, 2010: “Please note that largefile.txt file is created if it doesn’t exist. And largefile.txt file is overwritten if it exits.”
Last word should be “exists” :)
Reply
9 Dave February 24, 2010: Truncating brings up a good time to remind users about sparse files. Before you try to truncate a file, make sure the file isn’t a sparse file. With sparse files, checking the size with ls -l will NOT give you the true file size. In fact, the file may look many times larger than it really is.
Always check the file with the command du to check the size of the file if there is any chance you may be dealing with a sparse file or if you aren’t sure.
Reply
10 Philippe Petrinko February 24, 2010: @Dave: Yes, good to know/remember.
Nevertheless, [ls] can show blocks with [s / --size] option:
dd if=/dev/zero of=sparse bs=1 count=1 seek=1024k # let's create a sparse-file ls -ls sparse # show me your blocks 8 -rw-r--r-- 1 user1 user1 1048577 feb 24 18:38 sparse
Reply
11 Jay July 31, 2011: mentioning the ‘truncate’ command (as opposed to the library call), makes this Linux only (or other Unix-like without its own file utils, relying on GNU).
Reply