≡ Menu

HowTo: Debug Crashed Linux Application Core Files Like A Pro

Core dumps are often used to diagnose or debug errors in Linux or UNIX programs. Core dumps can serve as useful debugging aids for sys admins to find out why Application like Lighttpd, Apache, PHP-CGI or any other program crashed. Many vendors and open source project author requests a core file to troubleshoot a program. A core file is generated when an application program abnormally terminates due to bug, operating system security protection schema, or program simply try to write beyond the area of memory it has allocated, and so on. This article explains how to turn on core file support and track down bugs in programs.

Turn On Core File Creation Support

By default most Linux distributions turn off core file creation (at least this is true for RHEL, CentOS, Fedora and Suse Linux). You need to use the ulimit command to configure core files.

See The Current Core File Limits

Type the following command:
# ulimit -c
Sample outputs:


The output 0 (zero) means core file is not created.

Change Core File Limits

In this example, set the size limit of core files to 75000 bytes:
# ulimit -c 75000

HowTo: Enable Core File Dumps For Application Crashes And Segmentation Faults

Edit /etc/profile file and find line that read as follows to make persistent configuration:

ulimit -S -c 0 > /dev/null 2>&1

Update it as follows:

ulimit -c unlimited >/dev/null 2>&1

Save and close the file. Edit /etc/sysctl.conf, enter:
# vi /etc/sysctl.conf
Append the following lines:

kernel.core_uses_pid = 1
kernel.core_pattern = /tmp/core-%e-%s-%u-%g-%p-%t
fs.suid_dumpable = 2

Save and close the file. Where,

  1. kernel.core_uses_pid = 1 - Appends the coring processes PID to the core file name.
  2. fs.suid_dumpable = 2 - Make sure you get core dumps for setuid programs.
  3. kernel.core_pattern = /tmp/core-%e-%s-%u-%g-%p-%t - When the application terminates abnormally, a core file should appear in the /tmp. The kernel.core_pattern sysctl controls exact location of core file. You can define the core file name with the following template whih can contain % specifiers which are substituted by the following values when a core file is created:
    • %% - A single % character
    • %p - PID of dumped process
    • %u - real UID of dumped process
    • %g - real GID of dumped process
    • %s - number of signal causing dump
    • %t - time of dump (seconds since 0:00h, 1 Jan 1970)
    • %h - hostname (same as ’nodename’ returned by uname(2))
    • %e - executable filename

Finally, enable debugging for all apps, enter (Redhat and friends specific):
# echo "DAEMON_COREFILE_LIMIT='unlimited'" >> /etc/sysconfig/init
Reload the settings in /etc/sysctl.conf by running the following command:
# sysctl -p

How Do I Enable Core Dumping For Specific Deamon?

To enable core dumping for specific deamons, add the following line in the /etc/sysconfig/daemon-file file. In this example, edit /etc/init.d/lighttped and add line as follows:


Please note that DAEMON_COREFILE_LIMIT is Redhat specific, for all other distro add configuration as follows:

ulimit -c unlimited >/dev/null 2>&1
echo /tmp/core-%e-%s-%u-%g-%p-%t > /proc/sys/kernel/core_pattern

Save and close the file. Restart / reload lighttpd:
# /etc/init.d/lighttpd restart
# su - lighttpd
$ ulimit -c

Sample outputs:


Now, you can send core files to vendor or software writes.

How Do I Read Core Files?

You need use the gdb command as follows:
$ gdb /path/to/application /path/to/corefile
See the gdb command man page for more information.

strace command

System administrators, diagnosticians and trouble-shooters will find it invaluable for solving problems with programs for which the source is not readily available since they do not need to be recompiled in order to trace them. This is also useful to submit bug reports to open source developers. See how to use the strace command under Linux to debug the problems.

Recommended readings:

Stay tunned for gdb tutorial which will explains how to use generated core file to track down problem.

Tweet itFacebook itGoogle+ itPDF itFound an error/typo on this page?

{ 25 comments… add one }

  • shoaibi June 3, 2010, 10:12 am

    Great article, bravo :)

  • wonski June 3, 2010, 11:35 am

    pity there’s actually nothing about debugging..

    • nixCraft June 3, 2010, 11:44 am

      This part explained how to configure core dumps. I’m working on follow up article using gdb; so stay tuned.

      • neolix June 10, 2010, 6:01 am

        zahn pan tussi great ho topho kabul

  • storm June 3, 2010, 12:44 pm


    Your site is awesome especially for newbies and intermediates. Very clear and concise.


  • Tapas Mallick June 3, 2010, 1:36 pm

    “strace command” link at the end of the article is not correct and pointing to the current web page only.

    • nixCraft June 3, 2010, 5:16 pm

      Thanks for the heads up!

      @storm, thanks for the kind words!

  • Hitesh June 3, 2010, 5:55 pm

    Really nice info

  • nima0102 June 4, 2010, 4:39 pm

    Thanks for this article

  • Planet_Bob June 8, 2010, 3:23 am

    Nice article, just maybe make the subject line relevant to the article…

    “How to enable core dumps” instead of “Debug apps”…esp since there is virtually nothing about debugging in this article.

    • Anonymous June 9, 2010, 5:07 am


      Thanks for nice material, but i want to know how to analyze core dump
      thanks again

  • Jalal Hajigholamali June 9, 2010, 5:09 am


    thanks for nice article, but how can we analyze core dump ?
    thanks again

  • lokesh June 10, 2010, 11:42 am

    you send the linux commnad on my Email-ID

  • benzopilka July 1, 2010, 1:36 pm

    Please, explain more clear wich line add into /etc/sysconfig/daemon-file file, and which line into /etc/init.d/lighttped file; in the part about Specific Deamon?

  • Vijay Nayani August 23, 2010, 7:16 am

    Hello All,

    Anyboby ask me how do i read core files ???
    how to use gdb command ?????

  • BB April 26, 2011, 10:49 pm


    Just wondering why setting the following to “2” and not “1”?…

    fs.suid_dumpable = 2

  • Naveen July 5, 2011, 10:47 am


    i deleted the latest core file by mistake. now the core file is not getting created and it seems some functions like NTP is not working.

    How to solve this issue?

  • sachin July 13, 2011, 1:31 pm

    Anyone knows how to create Core Dump of entire System?

  • yaro March 4, 2012, 1:43 am

    Thanks for the information. Life becomes easy with people like you.

  • snapfla March 12, 2012, 4:11 pm

    Misleading title. This is not about debugging, much less, like a pro. It’s so frustrating when articles exist only to lure users that may click on ads.

  • moi August 28, 2012, 7:36 pm

    Do people really pay that much attention to titles? I came heree for the content, which was how to enable core dumps on linux. To that end, it worked like a charm.

  • Anthony Albert February 21, 2013, 1:31 am

    I’m returning to *nix programming after a long absence, and was trying to understand the use of DAEMON_COREFILE_LIMIT. What I haven’t seen mentioned is that this environment variable is used by the shell script function ‘daemon()’ which is included in the group of functions in etc/init.d/functions. If you start your daemon by calling daemon(), that variable will get used in a call to ‘ulimit’ by the shell. Otherwise, it will have no effect. I haven’t verified this yet (getting access to our servers is a slow process), but it seems that our custom shell script to start the daemon can do the same thing by calling ulimit directly.

  • xad February 21, 2013, 7:25 pm


  • alex April 26, 2013, 8:57 am

    root@duty-05-ubuntu:/var# echo “DAEMON_COREFILE_LIMIT=’unlimited'” >> /etc/sysconfig/init
    bash: /etc/sysconfig/init: No such file or directory

  • freeman June 19, 2013, 9:03 am

    Article does not tell how to actually does what the title says, and sadly google shows this as first page for actual topic in title.

Leave a Comment