Linux nginx: Chroot (Jail) Setup

by on April 6, 2010 · 20 comments· LAST UPDATED May 21, 2013

in , ,

How do I run Nginx web server in a chroot (jail) so that I can minimizes the damage done by a potential break-in by isolating the web server to a small section of the filesystem?

You can use traditional chroot kind of setup with nginx. Our sample setup:

  • Jail Directory : /nginx (D=/nginx)
  • Tested On : 64 Bit Linux Sytems (RHEL / CentOS / Fedora etc)
  • Nginx role : SSL and HTTP reverse proxy
  • Nginx 64 bit Libraries Path : /lib64 and /usr/lib64 (for 32 bit system use /lib and /usr/lib)

Step #1: Setup Chroot Directory

First, you need to define a chroot directory. Type the following commands:
# D=/nginx
# mkdir -p $D

Step #2: Create Isolated Environment

Type the following commands:
# mkdir -p $D/etc
# mkdir -p $D/dev
# mkdir -p $D/var
# mkdir -p $D/usr
# mkdir -p $D/usr/local/nginx
# mkdir -p $D/tmp
# chmod 1777 $D/tmp
# mkdir -p $D/var/tmp
# chmod 1777 $D/var/tmp
# mkdir -p $D/lib64

Step #3: Create Required Devices in $D/dev

You need to create the following three device entries so that nginx works without problem inside jail:
# ls -l /dev/{null,random,urandom}
Sample outputs:

crw-rw-rw- 1 root root 1, 3 Apr  5 11:03 /dev/null
crw-rw-rw- 1 root root 1, 8 Apr  5 11:03 /dev/random
cr--r--r-- 1 root root 1, 9 Apr  5 11:03 /dev/urandom

You need to use the mknod command to make block or character special files, enter:
# /bin/mknod -m 0666 $D/dev/null c 1 3
# /bin/mknod -m 0666 $D/dev/random c 1 8
# /bin/mknod -m 0444 $D/dev/urandom c 1 9

Step #4: Copy All Nginx Files In Directory

You need to copy /usr/local/nginx/ to $D/usr/local/nginx, enter:
# /bin/cp -farv /usr/local/nginx/* $D/usr/local/nginx

Step #5: Copy Required Libs To Jail

$D/usr/local/nginx/sbin/nginx depends upon various libraries, you need to copy them to $D/lib64 and $D/usr/lib64. To display shared library dependencies, enter:
# ldd /usr/local/nginx/sbin/nginx
Sample outputs:

	libpcre.so.0 => /lib64/libpcre.so.0 (0x000000316b800000)
	libssl.so.6 => /lib64/libssl.so.6 (0x0000003170400000)
	libcrypto.so.6 => /lib64/libcrypto.so.6 (0x000000316d400000)
	libdl.so.2 => /lib64/libdl.so.2 (0x000000316b000000)
	libz.so.1 => /usr/lib64/libz.so.1 (0x000000316c400000)
	libc.so.6 => /lib64/libc.so.6 (0x000000316ac00000)
	libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x000000316e400000)
	libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003170000000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x000000316ec00000)
	libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x000000316f800000)
	/lib64/ld-linux-x86-64.so.2 (0x000000316a800000)
	libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x000000316fc00000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x000000316f000000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x000000316d800000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x000000316c000000)
	libsepol.so.1 => /lib64/libsepol.so.1 (0x000000316bc00000)

You need to copy all of the above files to $D using the cp command as follows:
# cp /lib64/libsepol.so.1 $D/lib64
To automate this procedure use our script called n2chroot:
# cd /tmp
# wget http://bash.cyberciti.biz/dl/527.sh.zip
# unzip 527.sh.zip
# mv 527.sh /usr/bin/n2chroot
# chmod +x /usr/bin/n2chroot

Edit script and set BASE directory:
# vi /usr/bin/n2chroot
Finally, run it as follows:
# n2chroot /usr/local/nginx/sbin/nginx
# /bin/cp -fv /lib64/* $D/lib64

Step #6: Copy /etc To Jail

Finally, copy /etc to $D, enter:
# cp -fv /etc/{group,prelink.cache,services,adjtime,shells,gshadow,shadow,hosts.deny,localtime,nsswitch.conf,nscd.conf,prelink.conf,protocols,hosts,passwd,ld.so.cache,ld.so.conf,resolv.conf,host.conf} $D/etc
And a few directories too:
# cp -avr /etc/{ld.so.conf.d,prelink.conf.d} $D/etc

How Do I Start Chrooted nginx?

First, kill existing nginx (if running):
# killall -9 nginx
To start chrooted nginx, type:
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -t
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx

Make sure nginx starts when system reboots:
# echo '/usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx' >> /etc/rc.local

How Do I Reload Chrooted nginx?

Type the following command
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -s reload

How Do I Edit Chrooted nginx Configuration File?

Type the following commands:
# cd /nginx/usr/local/nginx/conf/
# vi nginx.conf

Save and close the file. Test and reload the same:
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -t
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -s reload

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 20 comments… read them below or add one }

1 Amr El-Sharnoby April 6, 2010 at 12:01 pm

Thanks a lot ;)

Reply

2 SW April 6, 2010 at 12:04 pm

Why
# mkdir -p $D/etc
# mkdir -p $D/dev
# mkdir -p $D/var
# mkdir -p $D/usr
instead of

# mkdir - p $D/{etc,dev,var}
?

Reply

3 Ryan Sharp December 24, 2011 at 10:12 pm

Why

"toMAYto"

instead of

"toMARto

?

To answer to your actual question without being as pointlessly pedantic as you are: because it makes absolutely no difference anyway and it’s POSIX compliant.

Reply

4 nixCraft April 6, 2010 at 10:18 pm

Yes, laziness on my part…

Reply

5 nhanth87 April 7, 2010 at 3:56 am

I want chroot nginx with php, my sql, etc… How can I do that ?

Reply

6 nixCraft April 8, 2010 at 5:27 pm

@ nhanth87,

Stay tunned for php-cgi chroot article :)

Reply

7 Erkan YILMAZ October 16, 2010 at 1:19 pm

Thanks, I had to tweak it sometimes, but in general it worked :-)

Debian (5.0r5) 32bit with nginx 0.8.52

Reply

8 Iain Kay November 23, 2010 at 5:18 pm

I tried this out on Ubuntu Server 10.04 (Lucid Lynx) and got an error. I had a good Google before posting this and did try the fix I found.

Error is:
[emerg]: getpwnam(“nobody”) failed in /usr/local/nginx/conf/nginx.conf:2

I copied /lib/libnss_* to /chroot/nginx/lib/ however this did not help.

Reply

9 nixCraft November 23, 2010 at 6:00 pm

getpwnam() get password file entry i.e. add nobody user to your jails $D/etc/passwd file.

Reply

10 Iain Kay November 23, 2010 at 6:45 pm

Hi Vivek,

I have already confirmed that the user exists in jails /etc directory, I have tried changing the UID/GID to below 100, over 100 and over 1000 but to no avail. I have tried permissions on all of the folders set to ‘nobody’ and set to ‘root’ with no avail.

At a bit of a loss here as to where it’s going wrong… I copied extra files pam.* from /etc to the jail directory also but not having it yet.

Reply

11 Iain Kay November 23, 2010 at 7:29 pm

I’m going to re-install the VM with Debian Squeeze and see how I get on with that. Got a sneaking suspicion that Ubuntu may be wreaking havoc on the /etc/passwd file with it’s customised authentication.
Whether it works or not will post back.

Reply

12 Iain Kay November 23, 2010 at 10:42 pm

Same problem when attempting to install this on Debian Lenny.

[emerg]: getpwnam(“nginx”) failed in /usr/local/nginx/conf/nginx.conf:2
configuration file /usr/local/nginx/conf/nginx.conf test failed

Definitely present in /chroot/nginx/etc/passwd + group + shadow.

Reply

13 wormik February 28, 2012 at 9:55 pm

well .. quite late – but found that this is due to missing libs – best is to execute via “strace” –

 strace -f -o /tmp/nginx.strace chroot /nginx /usr/local/nginx/sbin/nginx -t 

then you can easily track errors due to missing files (No such file kind of errors)

cheers

Reply

14 doublerebel January 12, 2011 at 9:15 pm

First of all, thanks for this excellent article and the related commands and scripts. Definitely made installation and troubleshooting much easier.

I successfully have nginx running chrooted on Ubuntu Server “Lucid Lynx” 10.04 LTS. It required several more directories created, paths were slightly different, and n2chroot missed two libs: libnss_compat.so.2 and libnsl.so.1.

To make this easier, I’ve created a script based on this blogpost and related scripts, and released it on my github:

https://github.com/doublerebel/nginx-chroot

Any updates/patches for Ubuntu Lucid or any other distro accepted!

Reply

15 Iain Kay November 23, 2010 at 10:58 pm

I fixed it by playing with libraries. On this install I had to run:
cp -Rv /lib/i686/cmov/* $D/lib/i686/cmov/

and then working. If I feel I have the effort I will go through and work out just the files I need but at least got it working.

Reply

16 M.S. Babaei April 4, 2011 at 12:18 am

tnx Vivek for such a useful and easy guide.

As I’m going to setup my new FreeBSD server + Nginx, is this guide works for FreeBSD?

As I understood chroot != jail, http://forums.freebsd.org/showthread.php?t=4356

Which way is better on BSD?

Reply

17 Søren November 26, 2011 at 9:45 pm

Hi Vivek and everyone,

Great tutorial – helped me to understand chroot :)
However, only reinvent the wheel, if you intend to learn more about wheels.

There are many high-level tools, that helps you create these jails.
Try out makejail or jailkit – it is MUUUCCH easier than doing the above!

In this case, I think many people ought to learn about the process going on in a chroot “jail” – but using the right tool can save you hundreds of hours in the long run (who said updating Nginx/php?)

Check out this tutorial (about Apache, but the process is very much the same).

Reply

18 ccspro February 10, 2012 at 12:07 am

Be careful what you include from the real /etc directory to your chroot dir…

->you do not need to copy the shadow file. (and possibly the group file)

->you do need a passwd file however you only (should) need to include the user(s) who are used to access those files within the chroot path. (so whatever user you use in the default config file as an example… mine is nginx) getpwnam() look ups seem to want this…

-> Another issue people may find is libraries are missing, ldd will only provide you with so many, to get a better picture of what is going on you can run ‘strace’ on the binary that is to be chrooted to find libraries that are ‘missing’ / cannot be found in the chroot lib path.

I’m using Centos 5.5 (as a base), however I’m using an newer vanilla kernel from kernel.org but built to work with my Centos layout, this will have no effect but worth mentioning…

Reply

19 bogusz14 January 24, 2013 at 8:04 am

I’m trying to do this in Centos 6.3 but there’s no “/usr/local/nginx” folder. Please help. Thanks a lot in advance.

Reply

20 Szépe Viktor March 12, 2014 at 9:52 pm

Leave a Comment

Tagged as:

Previous Faq:

Next Faq: