≡ Menu

linux kernel

How To Patch Running Linux Kernel Source Tree

Yesterday, I wrote about a serious Linux kernel bug and fix. However, few readers like to know about patching running Linux kernel. Patching production kernel is a risky business. Following procedure will help you to fix the problem.

Step # 1: Make sure your product is affected

First find out if your product is affected by reported exploit. For example, vmsplice() but only affects RHEL 5.x but RHEL 4.x,3.x, and 2.1.x are not affected at all. You can always obtain this information by visiting vendors bug reporting system called bugzilla. Also make sure bug affects your architectures. For example, a bug may only affect 64 bit or 32 bit platform.

Step # 2: Apply patch

You better apply and test patch in a test environment. Please note that some vendors such as Redhat and Suse modifies or backports kernel. So it is good idea to apply patch to their kernel source code tree. Otherwise you can always grab and apply patch to latest kernel version.

Step # 3: How do I apply kernel patch?

WARNING! These instructions require having the skills of a sysadmin. Personally, I avoid recompiling any kernel unless absolutely necessary. Most our production boxes (over 1400+) are powered by mix of RHEL 4 and 5. Wrong kernel option can disable hardware or may not boot system at all. If you don't understand the internal kernel dependencies don't try this on a production box.

Change directory to your kernel source code:
# cd linux-2.6.xx.yy
Download and save patch file as fix.vmsplice.exploit.patch:
# cat fix.vmsplice.exploit.patch

--- a/fs/splice.c
+++ b/fs/splice.c
@@ -1234,7 +1234,7 @@ static int get_iovec_page_array(const struct iovec __user *iov,
                if (unlikely(!len))
                error = -EFAULT;
-               if (unlikely(!base))
+               if (!access_ok(VERIFY_READ, base, len))

Now apply patch using patch command, enter:
# patch < fix.vmsplice.exploit.patch -p1
Now recompile and install Linux kernel.

I hope this quick and dirty guide will save someones time. On a related note Erek has unofficial patched RPMs for CentOS / RHEL distros.

Linux Kernel v2.6 Local Root Exploit ( vmsplice ) Found

Linux kernel version from 2.6.17 to all are affected because of vmsplice bug. The exploit code can be used to test if a kernel is vulnerable and it can start a root shell.

=> Debian Bug report logs

=> Fix 1 and Fix 2

Update: See how to apply a patch to kernel source tree.

How To Track Changes in Your Linux Filesystem

kfsmd is an interesting tool to keep track of changes in your filesystems. This tool based upon inotify which is a Linux kernel subsystem that provides file system event notification. Useful for file auditing. From the article:

Applications can ask the Linux kernel to report changes to selected files and directories. I created the Kernel Filesystem Monitoring Daemon (kfsmd) to make monitoring filesystem changes simple. Command-line clients for kfsmd come in two categories: monitoring and logging. The monitoring client produces output on the console whenever something happens to a filesystem you are watching. You can log to either a Berkeley DB4 file or a PostgreSQL database.

=> Use kfsmd to keep track of changes in your filesystems

Related: Linux audit files to see who made changes to a file

GPL v2.0 is Perfect for Linux – Says Linus Torvalds

Linux creator Linus Torvalds, in an interview being made public by the Linux Foundation Tuesday, stressed that version 2 of the GPL (GNU General Public License) still makes the most sense for the Linux kernel over the newer GPL version 3. Among GPL 3 highlights are protections against patent infringement lawsuits and provisions for license compatibility. Torvalds acknowledged he had spoken out against GPL 3 before it was released. He had opposed digital rights management provisions in early-2006, calling them burdensome.

Linus Torvalds, programmer, creator of the Linux kernel

On patent trolls, he says:

Yeah, they're kind of like the tourists that you can't bomb because there's nothing there to bomb. There are just these individuals that don't have anything to lose. That breaks the whole cold war model and seems to be one of the reasons that even big companies are now starting to realize that patents and software are a really bad idea.

The in-depth discussion has been split into two parts; the first segment is available today at Linux foundation blog. The next installment will be available in two weeks. Transcripts are also available on the LF website.

=> You can listen to complete conversations podcast here. If you'd rather read a transcript, you can find it here. (via Yahoo news - Image credit Wikipedia Linus article)

RHEL / CentOS Support 4GB or more RAM ( memory )

If you have 4 GB or more RAM use the Linux kernel compiled for PAE capable machines. Your machine may not show up total 4GB ram. All you have to do is install PAE kernel package.

This package includes a version of the Linux kernel with support for up to 64GB of high memory. It requires a CPU with Physical Address Extensions (PAE).
The non-PAE kernel can only address up to 4GB of memory. Install the kernel-PAE package if your machine has more than 4GB of memory (>=4GB).

How Do I Install PAE kernel?

To install PAE kernel, use yum command:
# yum install kernel-PAE

Loading "installonlyn" plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
Parsing package install arguments
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for kernel-PAE to pack into transaction set.
kernel-PAE-2.6.18-8.1.15. 100% |=========================| 207 kB    00:00
---> Package kernel-PAE.i686 0:2.6.18-8.1.15.el5 set to be installed
--> Running transaction check
Dependencies Resolved
 Package                 Arch       Version          Repository        Size
 kernel-PAE              i686       2.6.18-8.1.15.el5  updates            12 M
Transaction Summary
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 12 M
Is this ok [y/N]: y
Downloading Packages:
(1/1): kernel-PAE-2.6.18- 100% |=========================|  12 MB    00:12
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: kernel-PAE                   ######################### [1/1]
Installed: kernel-PAE.i686 0:2.6.18-8.1.15.el5

Just reboot the server and make sure you boot with PAE kernel i.e. 2.6.18-8.1.15.el5PAE:
# reboot

Download of the day: Linux kernel 2.6.23

Linux kernel version 2.6.23 has been released and available for download. Linus Torvalds writes:

Yeah, it got delayed, not because of any huge issues, but because of various bugfixes trickling in and causing me to reset my "release clock" all the time. But it's out there now, and hopefully better for the wait.Not a whole lot of changes since -rc9, although there's a few updates to mips, sparc64 and blackfin in there. Ignoring those arch updates, there's basically a number of mostly one-liners (mostly in drivers, but there's some networking fixes and soem VFS/VM fixes there too).

This version includes the new and shiny CFS process scheduler, a simpler read-ahead mechanism, the lguest 'Linux-on-Linux' paravirtualization hypervisor, XEN guest support, KVM smp guest support, and variable process argument length. SLUB is now the default slab allocator, there's SELinux protection for exploiting null dereferences using mmap, XFS and ext4 improvements, PPP over L2TP support. Also the 'lumpy' reclaim algorithm, a userspace driver framework, the O_CLOEXEC file descriptor flag, splice improvements, a new fallocate() syscall, lock statistics, support for multiqueue network devices, various new drivers, and many other minor features and fixes. See kernel change log here for more information.

Download Linux kernel version 2.6.23

=> Visit official Linux kernel web site here. See how to compile Linux kernel.

The stap program is the front-end to the Systemtap tool. It accepts probing instructions (written in a simple scripting language), translates those instructions into C code, compiles this C code, and loads the resulting kernel module into a running Linux kernel to perform the requested system trace/probe functions.

SystemTap provides free software (GPL) infrastructure to simplify the gathering of information about the running Linux system. This assists diagnosis of a performance or functional problem. SystemTap eliminates the need for the developer to go through the tedious and disruptive instrument, recompile, install, and reboot sequence that may be otherwise required to collect data.

We have several developers who use stap. Usually it works out of box. For example following program prints hello world on screen if SystemTap and related packages are installed:

stap -e 'probe begin { log ("hello world") }'

However under CentOS Linux version 5 (RHEL 5), you will get an error as follows:

semantic error: libdwfl failure (dwfl_linux_kernel_report_offline): No such file or directory while resolving probe point kernel.function("sys_*")

Install kernel-debuginfo package

To get rid of this problem, you have to simply install kernel-debuginfo package:
# yum install kernel-debuginfo
Please note that the installed kernel-debuginfo package must be for the same kernel release level and processor, so you may have to enter the following command:
# yum install kernel-debuginfo-KERNEL-VERSION-NUMBER

Hope this troubleshooting tip will help you out while working with systemtap (stap) scripts.