Comments on: Linux HugeTLBfs: Improve MySQL Database Application Performance http://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html This is a Linux sys admin journal by Vivek about sys admin work, Linux tips & tricks, hacks, news and more. Fri, 10 Feb 2012 20:37:43 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Gregoryhttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-168566 Gregory Fri, 04 Feb 2011 13:54:02 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-168566 @ Taylan Develioglu Right comment however I think these articles are supposed to be hints and then people should understand what and why and tune it to reflect their needs. Copying any configuration file/how to with no understanding (in general) is just the worst. @ Taylan Develioglu

Right comment however I think these articles are supposed to be hints and then people should understand what and why and tune it to reflect their needs. Copying any configuration file/how to with no understanding (in general) is just the worst.

]]> By: tariqhttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-157979 tariq Tue, 13 Jul 2010 14:31:44 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-157979 Hi , thanks for sharing the config guide on large page size for mysql . I followed your guide and had a few questions and was wondering if you would be kind enough to help me understand better .this is how my kernel parm. look like [root@gno-db-m04 /]# sysctl -p net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 vm.nr_hugepages = 512 vm.hugetlb_shm_group = 27 vm.nr_hugepages = 1024 i have set vm.nr_hugepages value to 512 and 1024 MB and added mysql group 27.NOW i am not able to figure out what should be the value of kernel.shmmax and kernel.shmall ? also this is what musql variable show . mysql> show variables like '%large%'; +---------------------+---------+ | Variable_name | Value | +---------------------+---------+ | large_files_support | ON | | large_page_size | 2097152 | | large_pages | ON | +---------------------+---------+ what does 2097152 indicate ? please help me understand . thank you in advance :) Hi , thanks for sharing the config guide on large page size for mysql . I followed your guide and had a few questions and was wondering if you would be kind enough to help me understand better .this is how my kernel parm. look like
[root@gno-db-m04 /]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
vm.nr_hugepages = 512
vm.hugetlb_shm_group = 27
vm.nr_hugepages = 1024

i have set vm.nr_hugepages value to 512 and 1024 MB and added mysql group 27.NOW i am not able to figure out what should be the value of kernel.shmmax and kernel.shmall ? also this is what musql variable show .

mysql> show variables like ‘%large%’;
+———————+———+
| Variable_name | Value |
+———————+———+
| large_files_support | ON |
| large_page_size | 2097152 |
| large_pages | ON |
+———————+———+
what does 2097152 indicate ? please help me understand . thank you in advance :)

]]> By: juanhttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-149829 juan Sun, 02 Aug 2009 16:02:31 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-149829 Can i use HugeTLBfs for PostgreSQL? Can i use HugeTLBfs for PostgreSQL?

]]> By: Taylan Develiogluhttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-149536 Taylan Develioglu Fri, 17 Jul 2009 12:58:52 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-149536 nonsense... have you noticed 68719476736 translates to 65gb? huge pages are only supported by the innodb engine , and vm.nr_hugepages should be set to match the memory used (i.e. buffer pool size) by it accordingly , not some arbitrary value as you're suggesting. And why raise the shared memory limit to 16gb if you're going to use only 1 gb (512 pages) (8gb is the default in linux 2.6). your post does not provide any useful information over what is already in the mysql documentation, and by omitting cricital details it is actually harmful. If you're going to blindly copy+paste existing documentation as if it was new, at least get it right. The mysql docs are at http://dev.mysql.com/doc/refman/5.1/en/large-page-support.html, for those interested nonsense…

have you noticed 68719476736 translates to 65gb?

huge pages are only supported by the innodb engine , and vm.nr_hugepages should be set to match the memory used (i.e. buffer pool size) by it accordingly , not some arbitrary value as you’re suggesting.

And why raise the shared memory limit to 16gb if you’re going to use only 1 gb (512 pages) (8gb is the default in linux 2.6).

your post does not provide any useful information over what is already in the mysql documentation, and by omitting cricital details it is actually harmful. If you’re going to blindly copy+paste existing documentation as if it was new, at least get it right.

The mysql docs are at http://dev.mysql.com/doc/refman/5.1/en/large-page-support.html, for those interested

]]> By: kim jong ilhttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-148729 kim jong il Wed, 27 May 2009 19:16:39 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-148729 this was news in 1986 when the '386 debuted.... this was news in 1986 when the ’386 debuted….

]]> By: wlonghttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-148659 wlong Fri, 22 May 2009 10:02:00 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-148659 I got an Innodb Error message: [root@andromeda ~]# tail -n 10 /var/log/mysqld.log 090523 01:58:57 mysqld started InnoDB: HugeTLB: Warning: Failed to allocate 8404992 bytes. errno 12 InnoDB HugeTLB: Warning: Using conventional memory pool 090523 1:58:57 InnoDB: Started; log sequence number 0 43655 Warning: Failed to allocate 8388608 bytes from HugeTLB memory. errno 12 Warning: Using conventional memory pool 090523 1:58:57 [Note] /usr/libexec/mysqld: ready for connections. Version: '5.0.45' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution I got an Innodb Error message:
[root@andromeda ~]# tail -n 10 /var/log/mysqld.log
090523 01:58:57 mysqld started
InnoDB: HugeTLB: Warning: Failed to allocate 8404992 bytes. errno 12
InnoDB HugeTLB: Warning: Using conventional memory pool
090523 1:58:57 InnoDB: Started; log sequence number 0 43655
Warning: Failed to allocate 8388608 bytes from HugeTLB memory. errno 12
Warning: Using conventional memory pool
090523 1:58:57 [Note] /usr/libexec/mysqld: ready for connections.
Version: ’5.0.45′ socket: ‘/var/lib/mysql/mysql.sock’ port: 3306 Source distribution

]]> By: Vivek Gitehttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-148633 Vivek Gite Wed, 20 May 2009 13:03:35 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-148633 The FreeBSD virtual memory subsystem now supports fully transparent use of superpages for application memory; application memory pages are dynamically promoted to or demoted from superpages without any modification to application code. This change offers the benefit of large page sizes such as improved virtual memory efficiency and reduced TLB (translation lookaside buffer) misses without downsides like application changes and virtual memory inflexibility. This is disabled by default and can be enabled by setting a loader tunable vm.pmap.pg_ps_enabled to 1. Add vm.pmap.pg_ps_enabled=1 to /boot/loader.conf. However, I've not tested it with MySQL or anything else... HTH The FreeBSD virtual memory subsystem now supports fully transparent use of superpages for application memory; application memory pages are dynamically promoted to or demoted from superpages without any modification to application code. This change offers the benefit of large page sizes such as improved virtual memory efficiency and reduced TLB (translation lookaside buffer) misses without downsides like application changes and virtual memory inflexibility. This is disabled by default and can be enabled by setting a loader tunable vm.pmap.pg_ps_enabled to 1. Add vm.pmap.pg_ps_enabled=1 to /boot/loader.conf.

However, I’ve not tested it with MySQL or anything else…

HTH

]]> By: Vladimir Micovichttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-148632 Vladimir Micovic Wed, 20 May 2009 12:53:02 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-148632 can i try this on fbsd? can i try this on fbsd?

]]> By: Vivek Gitehttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-148628 Vivek Gite Wed, 20 May 2009 09:26:03 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-148628 Yes, Ubuntu Linux has different file locations. Thanks for sharing the information. Yes, Ubuntu Linux has different file locations. Thanks for sharing the information.

]]> By: blink4bloghttp://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html#comment-148627 blink4blog Wed, 20 May 2009 08:52:42 +0000 http://www.cyberciti.biz/tips/?p=4773#comment-148627 I am implementing this under Ubuntu 9.0.4 with MySQL 5.0.75-0ubuntu10, few points I would like to clarify: 1. the command of "grep -i huge /proc/meminfo" returns the following: HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 <em>HugePages_Surp: 0</em> Hugepagesize: 4096 kB 2. my.cnf should be located under /etc/mysql/my.cnf 3. the command to restart mysql should be # /etc/init.d/mysql restart I am implementing this under Ubuntu 9.0.4 with MySQL 5.0.75-0ubuntu10, few points I would like to clarify:

1. the command of “grep -i huge /proc/meminfo” returns the following:

HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB

2. my.cnf should be located under /etc/mysql/my.cnf

3. the command to restart mysql should be # /etc/init.d/mysql restart

]]>