Linux incrond inotify: Monitor Directories For Changes And Take Action

by on July 18, 2010 · 23 comments· LAST UPDATED July 18, 2010

in

I want to copy (rsync to remote server) a directory tree whenever file uploaded or deleted in /var/www/html/upload/ directory under Linux operating systems for backup purpose and/or load balancing purpose without getting into complex file sharing setup such as NFS or GFS iscsi storage. How do I monitor /var/www/html/upload/ and its subdirectory for new files and executes rsync command to make copy back to www2.example.com:/var/www/html/upload/?

inotify is an inode-based filesystem notification technology. It provides possibility to simply monitor various events on files in filesystems. It is a very much powerful replacement of (obsolete) dnotify. inotify brings a comfortable way how to manage files used in your applications.

The incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to cron.

Install incron

Type the following command under RHEL / Fedora / CentOS Linux:
$ sudo yum install incron
Type the following command under Debian / Ubuntu Linux:
$ sudo apt-get install incron

Configuration Files

  • /etc/incron.conf - Main incron configuration file
  • /etc/incron.d/ - This directory is examined by incrond for system table files. You should put all your config file here as per directory or domain names.
  • /etc/incron.allow - This file contains users allowed to use incron.
  • /etc/incron.deny - This file contains users denied to use incron.
  • /var/spool/incron - This directory is examined by incrond for user table files which is set by users running the incrontab command.

incron Syntax

The syntax is as follows:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload

Where,

  • <directory> - It is nothing but path which is an absolute filesystem path such as /home/data. Any changes made to this path will result into command or action.
  • <file change mask> - Mask is is nothing but various file system events such as deleting a file. Each event can result into command execution. Use the following masks:
    • IN_ACCESS - File was accessed (read)
    • IN_ATTRIB - Metadata changed (permissions, timestamps, extended attributes, etc.)
    • IN_CLOSE_WRITE - File opened for writing was closed
    • IN_CLOSE_NOWRITE - File not opened for writing was closed
    • IN_CREATE - File/directory created in watched directory
    • IN_DELETE - File/directory deleted from watched directory
    • IN_DELETE_SELF - Watched file/directory was itself deleted
    • IN_MODIFY - File was modified
    • IN_MOVE_SELF - Watched file/directory was itself moved
    • IN_MOVED_FROM - File moved out of watched directory
    • IN_MOVED_TO - File moved into watched directory
    • IN_OPEN - File was opened
    • The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above events.
  • <command or action> - Run command or scripts when mask matched on given directory.
  • options - It can be any one of the following with command (i.e. you can pass it as arg to your command):
    1. $$ - dollar sign
    2. $@ - watched filesystem path (see above)
    3. $# - event-related file name
    4. $% - event flags (textually)
    5. $& - event flags (numerically)

Turn Service On

Type the following command:
# service incrond start
# chkconfig incrond on

Examples:

Type the following command to edit your incrontab
incrontab -e
Run logger command when file created or deleted from /tmp directory:
/tmp IN_ALL_EVENTS logger "/tmp action for $# file"
Save and close the file. Now cd to /tmp and create a file:
$ cd /tmp
$ >foo
$ rm foo

To see message, enter:
$ sudo tail -f /var/log/messages
Sample outputs:
Jul 17 18:39:25 vivek-desktop logger: "/tmp action for foo file"

How Do I Run Rsync Command To Replicate Files For /var/www/html/upload Directory?

Type the following command:
# incrontab -e
Append the following command:
/var/www/html/upload/ IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
Now, wherever files are uploaded in /var/www/html/upload/ directory, rsync will be executed to sync files to www2.example.com server. Make sure ssh keys are set for password less login.

How Do I Monitor /var/www/html/upload/ and Its Subdirectories Recursively?

You cannot monitor /var/www/html/upload/ directory recursively. However, you can use the find command to add all sub-directories as follows:
find /var/www/html/upload -type d -print0 | xargs -0 -I{} echo "{} IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/" > /etc/incron.d/webroot.conf
This will create /etc/incron.d/webroot.conf config as follows:

/var/www/html/upload IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/css IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/1 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/js IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/3 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/11 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/12 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/files IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/images IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/

How Do I Troubleshoot Problems?

You need to see /var/log/cron log file:
# tail -f /var/log/cron
# grep something /var/log/cron

Further readings:

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 23 comments… read them below or add one }

1 Nilesh July 18, 2010 at 4:48 am

Awesome man! I liked the most about rndc reload. Now no need to keep running rndc reload every hour or so.

Reply

2 nixCraft July 18, 2010 at 10:01 am

Why reload named every hour? Do you make any automated changes to zone file?

Reply

3 Nilesh July 18, 2010 at 2:56 pm

No I don’t. Just gave a thought. I know XName.org reloads it every hour. I don’t have too many zones.

Reply

4 Evaggelos Balaskas July 20, 2010 at 8:27 am

Hi, to resolve one of my biggest problem, i wrote a python program with inotify:
http://ebalaskas.gr./blog/?page=PIrsyncD

Reply

5 Evaggelos Balaskas July 20, 2010 at 8:29 am

Hi, nice article.

I’ve written a program based on inotify:

http://ebalaskas.gr./blog/?page=PIrsyncD

to resolve one of my biggest problems

Reply

6 Miraenda July 20, 2010 at 7:52 pm

None of my machines using the default CentOS repos seem to have incron available to install. Which repos are you using? My machines are Centos 5.5 ones (one 32-bit, one 64-bit).

Reply

7 nixCraft July 20, 2010 at 8:03 pm

It is from Fedora Project – EPEL repo. Install epel repo and you will get access to tons of binary packages.

rpm -qi incron | grep -i vendor
Version     : 0.5.5                             Vendor: Fedora Project

HTH

Reply

8 Miraenda July 20, 2010 at 8:06 pm

After adding EPEL per the tutorial you linked, I was able to add incron onto my machine. Thanks for the quick response and very useful information you’ve provided.

Reply

9 jjanks July 21, 2010 at 1:18 am

Silly, this can be done with a simple 4 or 5 line script in python.

hints
os.path.exists
os.unlink
popen

Reply

10 Nilesh July 21, 2010 at 5:43 am

Ya sure, but you need to run it as a daemon, it would be slow/cpuhog. incron is in C, so works much faster. There may be some python-inotify library which can be used instead of the above suggestion.
Also, using this method, you could write a bash daemon! (I have one on my local system to check that if pidgin runs, ensure skype runs, if pidgin doesn’t kill skype).

Reply

11 Nilesh July 21, 2010 at 6:28 am

And wohoo, here’s the python-inotify library- http://trac.dbzteam.org/pyinotify

Reply

12 Solaris July 21, 2010 at 9:29 am

Try that on tons of thousandths of files and you will see that your Python will perform in tens of seconds rather than milliseconds. You will say that there is no big difference but this will translate in disk reads a the physical level rather than operating system level triggers.

I speak from experience. I am not using incrond but something similar in production and it performs way faster that the similar Python code.

Use the right tool for the job.

Reply

13 Nilesh July 21, 2010 at 1:27 pm

That’s one of the libraries, there’s a C extension also available which directly handles with the kernel, i.e. responds to kernel events. Just google it out.

Reply

14 Tony July 22, 2010 at 5:06 pm

I don’t suppose we have similar version of this for Mac OS X ? Can’t seem to find one.

Reply

15 nixCraft July 23, 2010 at 1:01 pm

May be this will help. Edit: It appers that it is a work in progress software. If you know coding than you can always use knotify API.

Reply

16 Miraenda July 23, 2010 at 12:53 pm

From what I’m seeing online, the FreeBSD and Mac OS X implementation is called kqueue rather than inotify.

Reply

17 Miraenda July 23, 2010 at 1:19 pm

Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

(If this ends up double posting, apologies, but it didn’t appear to post or even be in moderation after I initially submitted it so re-adding it.)

Reply

18 nixCraft July 23, 2010 at 1:53 pm

DRBD is much better solution if you need to mirror entire partition in real time. It is like network based RAID-1.

Reply

19 Miraenda July 23, 2010 at 1:16 pm

Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

There’s a how-to on using it at Link

Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

Reply

20 Malik March 7, 2011 at 10:05 am

Hi all,

Is it possible to set incrontab in bash script? I tried to set normal crontab and it works without any problem but not “incrontab” :(

Any idea?

R

Reply

21 Rosco February 13, 2012 at 2:02 pm

This is easily the best website to check for new ideas. I was struggling with inotify for this kind of job and I wouldn’t have find this incron tool by myself.

Reply

22 Wojtek June 28, 2012 at 2:17 pm

Thanks for the tutorial. I was exactly looking for a tool like this one – thanks to this blog it even comes with a complete tutorial! :)

Reply

23 Reynold PJ May 11, 2013 at 5:52 am

Check out the script mentioned in the url pasted below. It uses inotifywait to recursively monitor and record file creation events.

http://jackal777.wordpress.com/2013/05/11/script-to-monitor-file-creation-under-all-cpanel-users-documentroot/

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: