≡ Menu

strace

Core dumps are often used to diagnose or debug errors in Linux or UNIX programs. Core dumps can serve as useful debugging aids for sys admins to find out why Application like Lighttpd, Apache, PHP-CGI or any other program crashed. Many vendors and open source project author requests a core file to troubleshoot a program. A core file is generated when an application program abnormally terminates due to bug, operating system security protection schema, or program simply try to write beyond the area of memory it has allocated, and so on. This article explains how to turn on core file support and track down bugs in programs.
[click to continue…]

According to wikipedia:

A segmentation fault occurs when a program attempts to access a memory location that it is not allowed to access, or attempts to access a memory location in a way that is not allowed (for example, attempting to write to a read-only location, or to overwrite part of the operating system).

Usually signal #11 (SIGSEGV) set, which is defined in the header file signal.h file. The default action for a program upon receiving SIGSEGV is abnormal termination. This action will end the process, but may generate a core file (also known as core dump) to aid debugging, or perform some other platform-dependent action. A core dump is the recorded state of the working memory of a computer program at a specific time, generally when the program has terminated abnormally.

Segmentation fault can also occur under following circumstances:

a) A buggy program / command, which can be only fixed by applying patch.

b) It can also appear when you try to access an array beyond the end of an array under C programming.

c) Inside a chrooted jail this can occur when critical shared libs, config file or /dev/ entry missing.

d) Sometime hardware or faulty memory or driver can also create problem.

e) Maintain suggested environment for all computer equipment (overheating can also generate this problem).

Suggestions to debug Segmentation Fault errors

To debug this kind of error try one or all of the following techniques :

  • Use gdb to track exact source of problem.
  • Make sure correct hardware installed and configured.
  • Always apply all patches and use updated system.
  • Make sure all dependencies installed inside jail.
  • Turn on core dumping for supported services such as Apache.
  • Use strace which is a useful diagnostic, instructional, and debugging tool.
  • Google and find out if there is a solution to problem.
  • Fix your C program for logical errors such as pointer, null pointer, arrays and so on.
  • Analyze core dump file generated by your system using gdb

Further readings:

Please add your suggestions and debugging techniques in the comment below.

strace is a useful diagnostic, instructional, and debugging tool. It can save lots of headache. System administrators, diagnosticians and trouble-shooters will find it invaluable for solving problems with programs for which the source is not readily available since they do not need to be recompiled in order to trace them. This is also useful to submit bug reports to open source developers.

Each line in the trace contains the system call name, followed by its arguments in parentheses and its return value.

Run strace against /bin/foo and capture its output to a text file in output.txt:
$ strace -o output.txt /bin/foo
You can strace the webserver process and see what it's doing. For example, strace php5 fastcgi process, enter:
$ strace -p 22254 -s 80 -o /tmp/debug.lighttpd.txt
To see only a trace of the open, read system calls, enter :
$ strace -e trace=open,read -p 22254 -s 80 -o debug.webserver.txt
Where,

  • -o filename : Write the trace output to the file filename rather than to screen (stderr).
  • -p PID : Attach to the process with the process ID pid and begin tracing. The trace may be terminated at any time by a keyboard interrupt signal (hit CTRL-C). strace will respond by detaching itself from the traced process(es) leaving it (them) to continue running. Multiple -p options can be used to attach to up to 32 processes in addition to command (which is optional if at least one -p option is given).
  • -s SIZE : Specify the maximum string size to print (the default is 32).

Refer to strace man page for more information:
$ man strace