You can use traditional chroot kind of setup with nginx. Our sample setup:
- Jail Directory : /nginx (D=/nginx)
- Tested On : 64 Bit Linux Sytems (RHEL / CentOS / Fedora etc)
- Nginx role : SSL and HTTP reverse proxy
- Nginx 64 bit Libraries Path : /lib64 and /usr/lib64 (for 32 bit system use /lib and /usr/lib)
Step #1: Setup Chroot Directory
First, you need to define a chroot directory. Type the following commands:
# D=/nginx
# mkdir -p $D
Step #2: Create Isolated Environment
Type the following commands:
# mkdir -p $D/etc
# mkdir -p $D/dev
# mkdir -p $D/var
# mkdir -p $D/usr
# mkdir -p $D/usr/local/nginx
# mkdir -p $D/tmp
# chmod 1777 $D/tmp
# mkdir -p $D/var/tmp
# chmod 1777 $D/var/tmp
# mkdir -p $D/lib64
Step #3: Create Required Devices in $D/dev
You need to create the following three device entries so that nginx works without problem inside jail:
# ls -l /dev/{null,random,urandom}
Sample outputs:
crw-rw-rw- 1 root root 1, 3 Apr 5 11:03 /dev/null crw-rw-rw- 1 root root 1, 8 Apr 5 11:03 /dev/random cr--r--r-- 1 root root 1, 9 Apr 5 11:03 /dev/urandom
You need to use the mknod command to make block or character special files, enter:
# /bin/mknod -m 0666 $D/dev/null c 1 3
# /bin/mknod -m 0666 $D/dev/random c 1 8
# /bin/mknod -m 0444 $D/dev/urandom c 1 9
Step #4: Copy All Nginx Files In Directory
You need to copy /usr/local/nginx/ to $D/usr/local/nginx, enter:
# /bin/cp -farv /usr/local/nginx/* $D/usr/local/nginx
Step #5: Copy Required Libs To Jail
$D/usr/local/nginx/sbin/nginx depends upon various libraries, you need to copy them to $D/lib64 and $D/usr/lib64. To display shared library dependencies, enter:
# ldd /usr/local/nginx/sbin/nginx
Sample outputs:
libpcre.so.0 => /lib64/libpcre.so.0 (0x000000316b800000) libssl.so.6 => /lib64/libssl.so.6 (0x0000003170400000) libcrypto.so.6 => /lib64/libcrypto.so.6 (0x000000316d400000) libdl.so.2 => /lib64/libdl.so.2 (0x000000316b000000) libz.so.1 => /usr/lib64/libz.so.1 (0x000000316c400000) libc.so.6 => /lib64/libc.so.6 (0x000000316ac00000) libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x000000316e400000) libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003170000000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x000000316ec00000) libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x000000316f800000) /lib64/ld-linux-x86-64.so.2 (0x000000316a800000) libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x000000316fc00000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x000000316f000000) libresolv.so.2 => /lib64/libresolv.so.2 (0x000000316d800000) libselinux.so.1 => /lib64/libselinux.so.1 (0x000000316c000000) libsepol.so.1 => /lib64/libsepol.so.1 (0x000000316bc00000)
You need to copy all of the above files to $D using the cp command as follows:
# cp /lib64/libsepol.so.1 $D/lib64
To automate this procedure use our script called n2chroot:
# cd /tmp
# wget http://bash.cyberciti.biz/dl/527.sh.zip
# unzip 527.sh.zip
# mv 527.sh /usr/bin/n2chroot
# chmod +x /usr/bin/n2chroot
Edit script and set BASE directory:
# vi /usr/bin/n2chroot
Finally, run it as follows:
# n2chroot /usr/local/nginx/sbin/nginx
# /bin/cp -fv /lib64/* $D/lib64
Step #6: Copy /etc To Jail
Finally, copy /etc to $D, enter:
# cp -fv /etc/{group,prelink.cache,services,adjtime,shells,gshadow,shadow,hosts.deny,localtime,nsswitch.conf,nscd.conf,prelink.conf,protocols,hosts,passwd,ld.so.cache,ld.so.conf,resolv.conf,host.conf} $D/etc
And a few directories too:
# cp -avr /etc/{ld.so.conf.d,prelink.conf.d} $D/etc
How Do I Start Chrooted nginx?
First, kill existing nginx (if running):
# killall -9 nginx
To start chrooted nginx, type:
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -t
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx
Make sure nginx starts when system reboots:
# echo '/usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx' >> /etc/rc.local
How Do I Reload Chrooted nginx?
Type the following command
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -s reload
How Do I Edit Chrooted nginx Configuration File?
Type the following commands:
# cd /nginx/usr/local/nginx/conf/
# vi nginx.conf
Save and close the file. Test and reload the same:
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -t
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -s reload
- CentOS / Redhat Linux: Install Keepalived To Provide IP Failover For Web Cluster
- CentOS / Redhat: Install nginx As Reverse Proxy Load Balancer
- Handling nginx Failover With KeepAlived
- nginx: Setup SSL Reverse Proxy (Load Balanced SSL Proxy)
- mod_extforward: Lighttpsd Log Clients Real IP Behind Reverse Proxy / Load Balancer
- HowTo: Merge Apache / Lighttpsd / Nginx Server Log Files
- Linux nginx: Chroot (Jail) Setup
- HowTo: SPDY SSL Installation and Configuration
- Install Nginx Using Yum Command on CentOS/RHEL
- Create a Self-Signed SSL Certificate on Nginx
Thanks a lot ;)
Why
instead of# mkdir -p $D/etc
# mkdir -p $D/dev
# mkdir -p $D/var
# mkdir -p $D/usr
?# mkdir - p $D/{etc,dev,var}
Why
instead of
?
To answer to your actual question without being as pointlessly pedantic as you are: because it makes absolutely no difference anyway and it’s POSIX compliant.
Because “# mkdir – p $D/{etc,dev,var}” misses “usr” :p
Yes, laziness on my part…
I want chroot nginx with php, my sql, etc… How can I do that ?
@ nhanth87,
Stay tunned for php-cgi chroot article :)
Thanks, I had to tweak it sometimes, but in general it worked :-)
Debian (5.0r5) 32bit with nginx 0.8.52
I tried this out on Ubuntu Server 10.04 (Lucid Lynx) and got an error. I had a good Google before posting this and did try the fix I found.
Error is:
[emerg]: getpwnam(“nobody”) failed in /usr/local/nginx/conf/nginx.conf:2
I copied /lib/libnss_* to /chroot/nginx/lib/ however this did not help.
getpwnam() get password file entry i.e. add nobody user to your jails $D/etc/passwd file.
Hi Vivek,
I have already confirmed that the user exists in jails /etc directory, I have tried changing the UID/GID to below 100, over 100 and over 1000 but to no avail. I have tried permissions on all of the folders set to ‘nobody’ and set to ‘root’ with no avail.
At a bit of a loss here as to where it’s going wrong… I copied extra files pam.* from /etc to the jail directory also but not having it yet.
I’m going to re-install the VM with Debian Squeeze and see how I get on with that. Got a sneaking suspicion that Ubuntu may be wreaking havoc on the /etc/passwd file with it’s customised authentication.
Whether it works or not will post back.
Same problem when attempting to install this on Debian Lenny.
[emerg]: getpwnam(“nginx”) failed in /usr/local/nginx/conf/nginx.conf:2
configuration file /usr/local/nginx/conf/nginx.conf test failed
Definitely present in /chroot/nginx/etc/passwd + group + shadow.
well .. quite late – but found that this is due to missing libs – best is to execute via “strace” –
then you can easily track errors due to missing files (No such file kind of errors)
cheers
First of all, thanks for this excellent article and the related commands and scripts. Definitely made installation and troubleshooting much easier.
I successfully have nginx running chrooted on Ubuntu Server “Lucid Lynx” 10.04 LTS. It required several more directories created, paths were slightly different, and n2chroot missed two libs: libnss_compat.so.2 and libnsl.so.1.
To make this easier, I’ve created a script based on this blogpost and related scripts, and released it on my github:
https://github.com/doublerebel/nginx-chroot
Any updates/patches for Ubuntu Lucid or any other distro accepted!
I fixed it by playing with libraries. On this install I had to run:
cp -Rv /lib/i686/cmov/* $D/lib/i686/cmov/
and then working. If I feel I have the effort I will go through and work out just the files I need but at least got it working.
tnx Vivek for such a useful and easy guide.
As I’m going to setup my new FreeBSD server + Nginx, is this guide works for FreeBSD?
As I understood chroot != jail, http://forums.freebsd.org/showthread.php?t=4356
Which way is better on BSD?
Hi Vivek and everyone,
Great tutorial – helped me to understand chroot :)
However, only reinvent the wheel, if you intend to learn more about wheels.
There are many high-level tools, that helps you create these jails.
Try out makejail or jailkit – it is MUUUCCH easier than doing the above!
In this case, I think many people ought to learn about the process going on in a chroot “jail” – but using the right tool can save you hundreds of hours in the long run (who said updating Nginx/php?)
Check out this tutorial (about Apache, but the process is very much the same).
Be careful what you include from the real /etc directory to your chroot dir…
->you do not need to copy the shadow file. (and possibly the group file)
->you do need a passwd file however you only (should) need to include the user(s) who are used to access those files within the chroot path. (so whatever user you use in the default config file as an example… mine is nginx) getpwnam() look ups seem to want this…
-> Another issue people may find is libraries are missing, ldd will only provide you with so many, to get a better picture of what is going on you can run ‘strace’ on the binary that is to be chrooted to find libraries that are ‘missing’ / cannot be found in the chroot lib path.
I’m using Centos 5.5 (as a base), however I’m using an newer vanilla kernel from kernel.org but built to work with my Centos layout, this will have no effect but worth mentioning…
I’m trying to do this in Centos 6.3 but there’s no “/usr/local/nginx” folder. Please help. Thanks a lot in advance.
https://gist.github.com/szepeviktor/9516285
Thanks for the article – concise and clear. But is there a point in chrooting Nginx? Its master process runs as root in order to bind to privileged ports, so chroot isn’t an obstacle for it. Am I wrong?