Linux nginx: Chroot (Jail) Setup

last updated in Categories , , ,

How do I run Nginx web server in a chroot (jail) so that I can minimizes the damage done by a potential break-in by isolating the web server to a small section of the filesystem?

You can use traditional chroot kind of setup with nginx. Our sample setup:

Advertisements

  • Jail Directory : /nginx (D=/nginx)
  • Tested On : 64 Bit Linux Sytems (RHEL / CentOS / Fedora etc)
  • Nginx role : SSL and HTTP reverse proxy
  • Nginx 64 bit Libraries Path : /lib64 and /usr/lib64 (for 32 bit system use /lib and /usr/lib)

Step #1: Setup Chroot Directory

First, you need to define a chroot directory. Type the following commands:
# D=/nginx
# mkdir -p $D

Step #2: Create Isolated Environment

Type the following commands:
# mkdir -p $D/etc
# mkdir -p $D/dev
# mkdir -p $D/var
# mkdir -p $D/usr
# mkdir -p $D/usr/local/nginx
# mkdir -p $D/tmp
# chmod 1777 $D/tmp
# mkdir -p $D/var/tmp
# chmod 1777 $D/var/tmp
# mkdir -p $D/lib64

Step #3: Create Required Devices in $D/dev

You need to create the following three device entries so that nginx works without problem inside jail:
# ls -l /dev/{null,random,urandom}
Sample outputs:

crw-rw-rw- 1 root root 1, 3 Apr  5 11:03 /dev/null
crw-rw-rw- 1 root root 1, 8 Apr  5 11:03 /dev/random
cr--r--r-- 1 root root 1, 9 Apr  5 11:03 /dev/urandom

You need to use the mknod command to make block or character special files, enter:
# /bin/mknod -m 0666 $D/dev/null c 1 3
# /bin/mknod -m 0666 $D/dev/random c 1 8
# /bin/mknod -m 0444 $D/dev/urandom c 1 9

Step #4: Copy All Nginx Files In Directory

You need to copy /usr/local/nginx/ to $D/usr/local/nginx, enter:
# /bin/cp -farv /usr/local/nginx/* $D/usr/local/nginx

Step #5: Copy Required Libs To Jail

$D/usr/local/nginx/sbin/nginx depends upon various libraries, you need to copy them to $D/lib64 and $D/usr/lib64. To display shared library dependencies, enter:
# ldd /usr/local/nginx/sbin/nginx
Sample outputs:

	libpcre.so.0 => /lib64/libpcre.so.0 (0x000000316b800000)
	libssl.so.6 => /lib64/libssl.so.6 (0x0000003170400000)
	libcrypto.so.6 => /lib64/libcrypto.so.6 (0x000000316d400000)
	libdl.so.2 => /lib64/libdl.so.2 (0x000000316b000000)
	libz.so.1 => /usr/lib64/libz.so.1 (0x000000316c400000)
	libc.so.6 => /lib64/libc.so.6 (0x000000316ac00000)
	libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x000000316e400000)
	libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003170000000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x000000316ec00000)
	libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x000000316f800000)
	/lib64/ld-linux-x86-64.so.2 (0x000000316a800000)
	libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x000000316fc00000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x000000316f000000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x000000316d800000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x000000316c000000)
	libsepol.so.1 => /lib64/libsepol.so.1 (0x000000316bc00000)

You need to copy all of the above files to $D using the cp command as follows:
# cp /lib64/libsepol.so.1 $D/lib64
To automate this procedure use our script called n2chroot:
# cd /tmp
# wget http://bash.cyberciti.biz/dl/527.sh.zip
# unzip 527.sh.zip
# mv 527.sh /usr/bin/n2chroot
# chmod +x /usr/bin/n2chroot
Edit script and set BASE directory:
# vi /usr/bin/n2chroot
Finally, run it as follows:
# n2chroot /usr/local/nginx/sbin/nginx
# /bin/cp -fv /lib64/* $D/lib64

Step #6: Copy /etc To Jail

Finally, copy /etc to $D, enter:
# cp -fv /etc/{group,prelink.cache,services,adjtime,shells,gshadow,shadow,hosts.deny,localtime,nsswitch.conf,nscd.conf,prelink.conf,protocols,hosts,passwd,ld.so.cache,ld.so.conf,resolv.conf,host.conf} $D/etc
And a few directories too:
# cp -avr /etc/{ld.so.conf.d,prelink.conf.d} $D/etc

How Do I Start Chrooted nginx?

First, kill existing nginx (if running):
# killall -9 nginx
To start chrooted nginx, type:
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -t
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx
Make sure nginx starts when system reboots:
# echo '/usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx' >> /etc/rc.local

How Do I Reload Chrooted nginx?

Type the following command
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -s reload

How Do I Edit Chrooted nginx Configuration File?

Type the following commands:
# cd /nginx/usr/local/nginx/conf/
# vi nginx.conf
Save and close the file. Test and reload the same:
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -t
# /usr/sbin/chroot /nginx /usr/local/nginx/sbin/nginx -s reload

ADVERTISEMENTS

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

22 comment

  2. Why
    # mkdir -p $D/etc
    # mkdir -p $D/dev
    # mkdir -p $D/var
    # mkdir -p $D/usr
     instead of

    # mkdir - p $D/{etc,dev,var}
    ?

    1. Why

      "toMAYto"

      instead of

      "toMARto

      ?

      To answer to your actual question without being as pointlessly pedantic as you are: because it makes absolutely no difference anyway and it’s POSIX compliant.

  6. Thanks, I had to tweak it sometimes, but in general it worked :-)

    Debian (5.0r5) 32bit with nginx 0.8.52

  7. I tried this out on Ubuntu Server 10.04 (Lucid Lynx) and got an error. I had a good Google before posting this and did try the fix I found.

    Error is:
    [emerg]: getpwnam(“nobody”) failed in /usr/local/nginx/conf/nginx.conf:2

    I copied /lib/libnss_* to /chroot/nginx/lib/ however this did not help.

      1. Hi Vivek,

        I have already confirmed that the user exists in jails /etc directory, I have tried changing the UID/GID to below 100, over 100 and over 1000 but to no avail. I have tried permissions on all of the folders set to ‘nobody’ and set to ‘root’ with no avail.

        At a bit of a loss here as to where it’s going wrong… I copied extra files pam.* from /etc to the jail directory also but not having it yet.

        1. I’m going to re-install the VM with Debian Squeeze and see how I get on with that. Got a sneaking suspicion that Ubuntu may be wreaking havoc on the /etc/passwd file with it’s customised authentication.
          Whether it works or not will post back.

          1. Same problem when attempting to install this on Debian Lenny.

            [emerg]: getpwnam(“nginx”) failed in /usr/local/nginx/conf/nginx.conf:2
            configuration file /usr/local/nginx/conf/nginx.conf test failed

            Definitely present in /chroot/nginx/etc/passwd + group + shadow.

            1. well .. quite late – but found that this is due to missing libs – best is to execute via “strace” –

               strace -f -o /tmp/nginx.strace chroot /nginx /usr/local/nginx/sbin/nginx -t

              then you can easily track errors due to missing files (No such file kind of errors)

              cheers

    2. First of all, thanks for this excellent article and the related commands and scripts. Definitely made installation and troubleshooting much easier.

      I successfully have nginx running chrooted on Ubuntu Server “Lucid Lynx” 10.04 LTS. It required several more directories created, paths were slightly different, and n2chroot missed two libs: libnss_compat.so.2 and libnsl.so.1.

      To make this easier, I’ve created a script based on this blogpost and related scripts, and released it on my github:

      https://github.com/doublerebel/nginx-chroot

      Any updates/patches for Ubuntu Lucid or any other distro accepted!

  8. I fixed it by playing with libraries. On this install I had to run:
    cp -Rv /lib/i686/cmov/* $D/lib/i686/cmov/

    and then working. If I feel I have the effort I will go through and work out just the files I need but at least got it working.

  10. Hi Vivek and everyone,

    Great tutorial – helped me to understand chroot :)
    However, only reinvent the wheel, if you intend to learn more about wheels.

    There are many high-level tools, that helps you create these jails.
    Try out makejail or jailkit – it is MUUUCCH easier than doing the above!

    In this case, I think many people ought to learn about the process going on in a chroot “jail” – but using the right tool can save you hundreds of hours in the long run (who said updating Nginx/php?)

    Check out this tutorial (about Apache, but the process is very much the same).

  11. Be careful what you include from the real /etc directory to your chroot dir…

    ->you do not need to copy the shadow file. (and possibly the group file)

    ->you do need a passwd file however you only (should) need to include the user(s) who are used to access those files within the chroot path. (so whatever user you use in the default config file as an example… mine is nginx) getpwnam() look ups seem to want this…

    -> Another issue people may find is libraries are missing, ldd will only provide you with so many, to get a better picture of what is going on you can run ‘strace’ on the binary that is to be chrooted to find libraries that are ‘missing’ / cannot be found in the chroot lib path.

    I’m using Centos 5.5 (as a base), however I’m using an newer vanilla kernel from kernel.org but built to work with my Centos layout, this will have no effect but worth mentioning…

    1. I’m trying to do this in Centos 6.3 but there’s no “/usr/local/nginx” folder. Please help. Thanks a lot in advance.

  13. Thanks for the article – concise and clear. But is there a point in chrooting Nginx? Its master process runs as root in order to bind to privileged ports, so chroot isn’t an obstacle for it. Am I wrong?

    Still, have a question? Get help on our forum!

Tagged as: Tags