≡ Menu

Linux incrond inotify: Monitor Directories For Changes And Take Action

I want to copy (rsync to remote server) a directory tree whenever file uploaded or deleted in /var/www/html/upload/ directory under Linux operating systems for backup purpose and/or load balancing purpose without getting into complex file sharing setup such as NFS or GFS iscsi storage. How do I monitor /var/www/html/upload/ and its subdirectory for new files and executes rsync command to make copy back to www2.example.com:/var/www/html/upload/?

inotify is an inode-based filesystem notification technology. It provides possibility to simply monitor various events on files in filesystems. It is a very much powerful replacement of (obsolete) dnotify. inotify brings a comfortable way how to manage files used in your applications.

The incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to cron.

Install incron

Type the following command under RHEL / Fedora / CentOS Linux:
$ sudo yum install incron
Type the following command under Debian / Ubuntu Linux:
$ sudo apt-get install incron

Configuration Files

  • /etc/incron.conf - Main incron configuration file
  • /etc/incron.d/ - This directory is examined by incrond for system table files. You should put all your config file here as per directory or domain names.
  • /etc/incron.allow - This file contains users allowed to use incron.
  • /etc/incron.deny - This file contains users denied to use incron.
  • /var/spool/incron - This directory is examined by incrond for user table files which is set by users running the incrontab command.

incron Syntax

The syntax is as follows:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload

Where,

  • <directory> - It is nothing but path which is an absolute filesystem path such as /home/data. Any changes made to this path will result into command or action.
  • <file change mask> - Mask is is nothing but various file system events such as deleting a file. Each event can result into command execution. Use the following masks:
    • IN_ACCESS - File was accessed (read)
    • IN_ATTRIB - Metadata changed (permissions, timestamps, extended attributes, etc.)
    • IN_CLOSE_WRITE - File opened for writing was closed
    • IN_CLOSE_NOWRITE - File not opened for writing was closed
    • IN_CREATE - File/directory created in watched directory
    • IN_DELETE - File/directory deleted from watched directory
    • IN_DELETE_SELF - Watched file/directory was itself deleted
    • IN_MODIFY - File was modified
    • IN_MOVE_SELF - Watched file/directory was itself moved
    • IN_MOVED_FROM - File moved out of watched directory
    • IN_MOVED_TO - File moved into watched directory
    • IN_OPEN - File was opened
    • The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above events.
  • <command or action> - Run command or scripts when mask matched on given directory.
  • options - It can be any one of the following with command (i.e. you can pass it as arg to your command):
    1. $$ - dollar sign
    2. $@ - watched filesystem path (see above)
    3. $# - event-related file name
    4. $% - event flags (textually)
    5. $& - event flags (numerically)

Turn Service On

Type the following command:
# service incrond start
# chkconfig incrond on

Examples:

Type the following command to edit your incrontab
incrontab -e
Run logger command when file created or deleted from /tmp directory:
/tmp IN_ALL_EVENTS logger "/tmp action for $# file"
Save and close the file. Now cd to /tmp and create a file:
$ cd /tmp
$ >foo
$ rm foo

To see message, enter:
$ sudo tail -f /var/log/messages
Sample outputs:
Jul 17 18:39:25 vivek-desktop logger: "/tmp action for foo file"

How Do I Run Rsync Command To Replicate Files For /var/www/html/upload Directory?

Type the following command:
# incrontab -e
Append the following command:
/var/www/html/upload/ IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
Now, wherever files are uploaded in /var/www/html/upload/ directory, rsync will be executed to sync files to www2.example.com server. Make sure ssh keys are set for password less login.

How Do I Monitor /var/www/html/upload/ and Its Subdirectories Recursively?

You cannot monitor /var/www/html/upload/ directory recursively. However, you can use the find command to add all sub-directories as follows:
find /var/www/html/upload -type d -print0 | xargs -0 -I{} echo "{} IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/" > /etc/incron.d/webroot.conf
This will create /etc/incron.d/webroot.conf config as follows:

/var/www/html/upload IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/css IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/1 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/js IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/3 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/11 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/12 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/files IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/images IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/

How Do I Troubleshoot Problems?

You need to see /var/log/cron log file:
# tail -f /var/log/cron
# grep something /var/log/cron

Further readings:

Tweet itFacebook itGoogle+ itPDF itFound an error/typo on this page?

{ 28 comments… add one }

  • Nilesh July 18, 2010, 4:48 am

    Awesome man! I liked the most about rndc reload. Now no need to keep running rndc reload every hour or so.

    • nixCraft July 18, 2010, 10:01 am

      Why reload named every hour? Do you make any automated changes to zone file?

      • Nilesh July 18, 2010, 2:56 pm

        No I don’t. Just gave a thought. I know XName.org reloads it every hour. I don’t have too many zones.

  • Evaggelos Balaskas July 20, 2010, 8:27 am

    Hi, to resolve one of my biggest problem, i wrote a python program with inotify:
    http://ebalaskas.gr./blog/?page=PIrsyncD

  • Evaggelos Balaskas July 20, 2010, 8:29 am

    Hi, nice article.

    I’ve written a program based on inotify:

    http://ebalaskas.gr./blog/?page=PIrsyncD

    to resolve one of my biggest problems

  • Miraenda July 20, 2010, 7:52 pm

    None of my machines using the default CentOS repos seem to have incron available to install. Which repos are you using? My machines are Centos 5.5 ones (one 32-bit, one 64-bit).

    • nixCraft July 20, 2010, 8:03 pm

      It is from Fedora Project – EPEL repo. Install epel repo and you will get access to tons of binary packages.

      rpm -qi incron | grep -i vendor
      Version     : 0.5.5                             Vendor: Fedora Project
      

      HTH

      • Miraenda July 20, 2010, 8:06 pm

        After adding EPEL per the tutorial you linked, I was able to add incron onto my machine. Thanks for the quick response and very useful information you’ve provided.

  • jjanks July 21, 2010, 1:18 am

    Silly, this can be done with a simple 4 or 5 line script in python.

    hints
    os.path.exists
    os.unlink
    popen

    • Nilesh July 21, 2010, 5:43 am

      Ya sure, but you need to run it as a daemon, it would be slow/cpuhog. incron is in C, so works much faster. There may be some python-inotify library which can be used instead of the above suggestion.
      Also, using this method, you could write a bash daemon! (I have one on my local system to check that if pidgin runs, ensure skype runs, if pidgin doesn’t kill skype).

    • Solaris July 21, 2010, 9:29 am

      Try that on tons of thousandths of files and you will see that your Python will perform in tens of seconds rather than milliseconds. You will say that there is no big difference but this will translate in disk reads a the physical level rather than operating system level triggers.

      I speak from experience. I am not using incrond but something similar in production and it performs way faster that the similar Python code.

      Use the right tool for the job.

      • Nilesh July 21, 2010, 1:27 pm

        That’s one of the libraries, there’s a C extension also available which directly handles with the kernel, i.e. responds to kernel events. Just google it out.

  • Tony July 22, 2010, 5:06 pm

    I don’t suppose we have similar version of this for Mac OS X ? Can’t seem to find one.

  • Miraenda July 23, 2010, 12:53 pm

    From what I’m seeing online, the FreeBSD and Mac OS X implementation is called kqueue rather than inotify.

    • Miraenda July 23, 2010, 1:19 pm

      Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

      Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

      (If this ends up double posting, apologies, but it didn’t appear to post or even be in moderation after I initially submitted it so re-adding it.)

      • nixCraft July 23, 2010, 1:53 pm

        DRBD is much better solution if you need to mirror entire partition in real time. It is like network based RAID-1.

  • Miraenda July 23, 2010, 1:16 pm

    Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

    There’s a how-to on using it at Link

    Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

  • Malik March 7, 2011, 10:05 am

    Hi all,

    Is it possible to set incrontab in bash script? I tried to set normal crontab and it works without any problem but not “incrontab” :(

    Any idea?

    R

  • Rosco February 13, 2012, 2:02 pm

    This is easily the best website to check for new ideas. I was struggling with inotify for this kind of job and I wouldn’t have find this incron tool by myself.

  • Wojtek June 28, 2012, 2:17 pm

    Thanks for the tutorial. I was exactly looking for a tool like this one – thanks to this blog it even comes with a complete tutorial! :)

  • Reynold PJ May 11, 2013, 5:52 am

    Check out the script mentioned in the url pasted below. It uses inotifywait to recursively monitor and record file creation events.

    http://jackal777.wordpress.com/2013/05/11/script-to-monitor-file-creation-under-all-cpanel-users-documentroot/

  • lemon May 4, 2014, 6:31 pm

    HI Nix craft,

    when i use incrontab out of the box, i mean no changes,

    i have no result, as if the cmd was not executed.

    my incron line is :

    /crrae IN_CREATE,IN_CLOSE_WRITE,IN_MOVED_TO /usr/bin/unix2dos $#

    the log give information as if everything was ok but no actually

    when i change my incron to :

    /crrae IN_CREATE,IN_CLOSE_WRITE,IN_MOVED_TO /usr/bin/unix2dos /crrae/$#

    the log give me this:

    May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpPbqoFd)
    May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpN2Odxf)
    May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpN2Odxf)
    

    multiple times

    here is the incron.conf:

    [root@srvovs ~]# cat /etc/incron.conf
    #
    # *** incron example configuration file ***
    #
    # (c) Lukas Jelinek, 2007, 2008
    #
    # Parameter:   system_table_dir
    # Meaning:     system table directory
    # Description: This directory is examined by incrond for system table files.
    # Default:     /etc/incron.d
    #
    # Example:
    # system_table_dir = /var/spool/incron.systables
    # Parameter:   user_table_dir
    # Meaning:     user table directory
    # Description: This directory is examined by incrond for user table files.
    # Default:     /var/spool/incron
    #
    # Example:
    # user_table_dir = /var/spool/incron.usertables
    # Parameter:   allowed_users
    # Meaning:     allowed users list file
    # Description: This file contains users allowed to use incron.
    # Default:     /etc/incron.allow
    #
    # Example:
    # allowed_users = /etc/incron/allow
    # Parameter:   denied_users
    # Meaning:     denied users list file
    # Description: This file contains users denied to use incron.
    # Default:     /etc/incron.deny
    #
    # Example:
    # denied_users = /etc/incron/deny
    # Parameter:   lockfile_dir
    # Meaning:     application lock file directory
    # Description: This directory is used for creating a lock avoiding to run
    #              multiple instances of incrond.
    # Default:     /var/run
    #
    # Example:
    # lockfile_dir = /tmp
    # Parameter:   lockfile_name
    # Meaning:     application lock file name base
    # Description: This name (appended by '.pid') is used for creating a lock
    #              avoiding to run multiple instances of incrond.
    # Default:     incrond
    #
    # Example:
    # lockfile_name = incron.lock
    # Parameter:   editor
    # Meaning:     editor executable
    # Description: This name or path is used to run as an editor for editting
    #              user tables.
    # Default:     vim
    #
    # Example:
    # editor = nano
    

    any idea ?

  • menensky June 13, 2014, 11:51 pm

    Hi Nix craft,
    This is helpful I appended an email notification whenever a file is being copied but when there’s a lot of file being transferred its spamming my email because it is monitoring every event.. Any suggestion to send email per batch not per files.

  • Ishai August 3, 2014, 11:21 am

    i have a question,

    im running cuckoo sandbox,
    and i want to make a folder, that whenever i put a sample there, this script will perform the next action

    python submit.py

    how could i do that?

  • jakedemus May 23, 2015, 2:11 pm

    Has anyone got incron & unison successfully working together to sync remote servers, all on CentOS 6? If so, can you explain, please?

    Thanks

    • stevep June 30, 2015, 7:57 am

      We use this line for “incrontab -e”:

      /home/user-to-be-replicated/base-directory IN_CREATE,IN_DELETE,IN_CLOSE_WRITE unison -batch -group -owner /home/user-to-be-replicated/base-directory ssh://otherhost//home/user-to-be-replicated/base-directory

      Additionally we configured password-less auth for root (private/public-key pair) and enabled incron for root:
      echo ‘root’ >> /etc/incron.allow

      I hope this helps.

Leave a Comment