Linux HugeTLBfs: Improve MySQL Database Application Performance

Posted on in Categories CentOS, Hardware, High performance computing, Howto, MySQL, RedHat/Fedora Linux last updated May 20, 2009

Applications that perform a lot of memory accesses (several GBs) may obtain performance improvements by using large pages due to reduced Translation Lookaside Buffer (TLB) misses. HugeTLBfs is memory management feature offered in Linux kernel, which is valuable for applications that use a large virtual address space. It is especially useful for database applications such as MySQL, Oracle and others. Other server software(s) that uses the prefork or similar (e.g. Apache web server) model will also benefit.

The CPU’s Translation Lookaside Buffer (TLB) is a small cache used for storing virtual-to-physical mapping information. By using the TLB, a translation can be performed without referencing the in-memory page table entry that maps the virtual address. However, to keep translations as fast as possible, the TLB is usually small. It is not uncommon for large memory applications to exceed the mapping capacity of the TLB. Users can use the huge page support in Linux kernel by either using the mmap system call or standard SYSv shared memory system calls (shmget, shmat).

valgrind – Linux Tools For Debugging And Profiling Programs ( bug reporting tool )

Posted on in Categories Hardware, Howto, Linux, Sys admin, Troubleshooting last updated April 15, 2008

Few days back I wrote about strace tool for reporting and finding bug in program. Today I’m going to talk about another interesting tool called valgrind.

Valgrind is a flexible program for debugging and profiling Linux executables. It consists of a core, which provides a synthetic CPU in software, and a series of “tools”, each of which is a debugging or profiling tool. The architecture is modular, so that new tools can be created easily and without disturbing the existing structure. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.

The Valgrind distribution currently includes five production-quality tools:

  • a memory error detector
  • a thread error detector
  • a cache and branch-prediction profiler
  • a call-graph generating cache profiler
  • a heap profiler

It also includes two experimental tools:

  • a data race detector
  • an instant memory leak detector.

It runs on the following platforms:

  • X86/Linux
  • AMD64/Linux
  • PPC32/Linux
  • PPC64/Linux

How do I use valgrind?

Valgrind is typically run as follows:
$ valgrind command-name arg1 arg2 argN
$ valgrind program args
$ valgrind ./myapp -d /tmp -f 120

You can select tool using the –tool=TOOLName option. For example use memcheck which is a fine-grained memory checker. To generate trace back for command called myapp, enter:
$ valgrind --tool=memcheck -v --log-file=myapp.dbg --num-callers=8 ./myapp -d /tmp -f 120
Where,

  • –tool=memcheck : Run the Valgrind tool called memcheck
  • -v : Verbose output
  • –log-file=myapp.dbg : Specifies that Valgrind should send all of its messages to the specified file.
  • –num-callers=8 : By default, Valgrind shows twelve levels of function call names to help you identify program locations. You can change that number with this option. This can help in determining the program’s location in deeply-nested call chains.

The –leak-check option turns on the detailed memory leak detector:
$ valgrind --tool=memcheck -v --log-file=myapp.dbg --num-callers=8 --leak-check=yes ./myapp -d /tmp -f 120

Further readings: