Linux incrond inotify: Monitor Directories For Changes And Take Action

Posted on in Categories last updated July 18, 2010

I want to copy (rsync to remote server) a directory tree whenever file uploaded or deleted in /var/www/html/upload/ directory under Linux operating systems for backup purpose and/or load balancing purpose without getting into complex file sharing setup such as NFS or GFS iscsi storage. How do I monitor /var/www/html/upload/ and its subdirectory for new files and executes rsync command to make copy back to www2.example.com:/var/www/html/upload/?

inotify is an inode-based filesystem notification technology. It provides possibility to simply monitor various events on files in filesystems. It is a very much powerful replacement of (obsolete) dnotify. inotify brings a comfortable way how to manage files used in your applications.

The incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to cron.

Install incron

Type the following command under RHEL / Fedora / CentOS Linux:
$ sudo yum install incron
Type the following command under Debian / Ubuntu Linux:
$ sudo apt-get install incron

Configuration Files

  • /etc/incron.conf – Main incron configuration file
  • /etc/incron.d/ – This directory is examined by incrond for system table files. You should put all your config file here as per directory or domain names.
  • /etc/incron.allow – This file contains users allowed to use incron.
  • /etc/incron.deny – This file contains users denied to use incron.
  • /var/spool/incron – This directory is examined by incrond for user table files which is set by users running the incrontab command.

incron Syntax

The syntax is as follows:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload

Where,

  • <directory> – It is nothing but path which is an absolute filesystem path such as /home/data. Any changes made to this path will result into command or action.
  • <file change mask> – Mask is is nothing but various file system events such as deleting a file. Each event can result into command execution. Use the following masks:
    • IN_ACCESS – File was accessed (read)
    • IN_ATTRIB – Metadata changed (permissions, timestamps, extended attributes, etc.)
    • IN_CLOSE_WRITE – File opened for writing was closed
    • IN_CLOSE_NOWRITE – File not opened for writing was closed
    • IN_CREATE – File/directory created in watched directory
    • IN_DELETE – File/directory deleted from watched directory
    • IN_DELETE_SELF – Watched file/directory was itself deleted
    • IN_MODIFY – File was modified
    • IN_MOVE_SELF – Watched file/directory was itself moved
    • IN_MOVED_FROM – File moved out of watched directory
    • IN_MOVED_TO – File moved into watched directory
    • IN_OPEN – File was opened
    • The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above events.
  • <command or action> – Run command or scripts when mask matched on given directory.
  • options – It can be any one of the following with command (i.e. you can pass it as arg to your command):
    1. $$ – dollar sign
    2. $@ – watched filesystem path (see above)
    3. $# – event-related file name
    4. $% – event flags (textually)
    5. $& – event flags (numerically)

Turn Service On

Type the following command:
# service incrond start
# chkconfig incrond on

Examples:

Type the following command to edit your incrontab
incrontab -e
Run logger command when file created or deleted from /tmp directory:
/tmp IN_ALL_EVENTS logger "/tmp action for $# file"
Save and close the file. Now cd to /tmp and create a file:
$ cd /tmp
$ >foo
$ rm foo
To see message, enter:
$ sudo tail -f /var/log/messages
Sample outputs:
Jul 17 18:39:25 vivek-desktop logger: "/tmp action for foo file"

How Do I Run Rsync Command To Replicate Files For /var/www/html/upload Directory?

Type the following command:
# incrontab -e
Append the following command:
/var/www/html/upload/ IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
Now, wherever files are uploaded in /var/www/html/upload/ directory, rsync will be executed to sync files to www2.example.com server. Make sure ssh keys are set for password less login.

How Do I Monitor /var/www/html/upload/ and Its Subdirectories Recursively?

You cannot monitor /var/www/html/upload/ directory recursively. However, you can use the find command to add all sub-directories as follows:
find /var/www/html/upload -type d -print0 | xargs -0 -I{} echo "{} IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/" > /etc/incron.d/webroot.conf
This will create /etc/incron.d/webroot.conf config as follows:

/var/www/html/upload IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/css IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/1 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/js IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/3 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/2010 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/2010/11 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/2010/12 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/2 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/files IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/
/var/www/html/upload/images IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ [email protected]:/var/www/html/upload/

How Do I Troubleshoot Problems?

You need to see /var/log/cron log file:
# tail -f /var/log/cron
# grep something /var/log/cron

Further readings:

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

Share this on (or read 32 comments/add one below):

32 comment

  4. None of my machines using the default CentOS repos seem to have incron available to install. Which repos are you using? My machines are Centos 5.5 ones (one 32-bit, one 64-bit).

    Reply Report comment

      1. After adding EPEL per the tutorial you linked, I was able to add incron onto my machine. Thanks for the quick response and very useful information you’ve provided.

        Reply Report comment

    1. Ya sure, but you need to run it as a daemon, it would be slow/cpuhog. incron is in C, so works much faster. There may be some python-inotify library which can be used instead of the above suggestion.
      Also, using this method, you could write a bash daemon! (I have one on my local system to check that if pidgin runs, ensure skype runs, if pidgin doesn’t kill skype).

      Reply Report comment

    2. Try that on tons of thousandths of files and you will see that your Python will perform in tens of seconds rather than milliseconds. You will say that there is no big difference but this will translate in disk reads a the physical level rather than operating system level triggers.

      I speak from experience. I am not using incrond but something similar in production and it performs way faster that the similar Python code.

      Use the right tool for the job.

      Reply Report comment

    1. Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

      Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

      (If this ends up double posting, apologies, but it didn’t appear to post or even be in moderation after I initially submitted it so re-adding it.)

      Reply Report comment

  8. Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

    There’s a how-to on using it at Link

    Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

    Reply Report comment

  13. HI Nix craft,

    when i use incrontab out of the box, i mean no changes,

    i have no result, as if the cmd was not executed.

    my incron line is :

    /crrae IN_CREATE,IN_CLOSE_WRITE,IN_MOVED_TO /usr/bin/unix2dos $#

    the log give information as if everything was ok but no actually

    when i change my incron to :

    /crrae IN_CREATE,IN_CLOSE_WRITE,IN_MOVED_TO /usr/bin/unix2dos /crrae/$#

    the log give me this:

    May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpPbqoFd)
May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpN2Odxf)
May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpN2Odxf)

    multiple times

    here is the incron.conf:

    [[email protected] ~]# cat /etc/incron.conf
#
# *** incron example configuration file ***
#
# (c) Lukas Jelinek, 2007, 2008
#


# Parameter:   system_table_dir
# Meaning:     system table directory
# Description: This directory is examined by incrond for system table files.
# Default:     /etc/incron.d
#
# Example:
# system_table_dir = /var/spool/incron.systables


# Parameter:   user_table_dir
# Meaning:     user table directory
# Description: This directory is examined by incrond for user table files.
# Default:     /var/spool/incron
#
# Example:
# user_table_dir = /var/spool/incron.usertables


# Parameter:   allowed_users
# Meaning:     allowed users list file
# Description: This file contains users allowed to use incron.
# Default:     /etc/incron.allow
#
# Example:
# allowed_users = /etc/incron/allow


# Parameter:   denied_users
# Meaning:     denied users list file
# Description: This file contains users denied to use incron.
# Default:     /etc/incron.deny
#
# Example:
# denied_users = /etc/incron/deny


# Parameter:   lockfile_dir
# Meaning:     application lock file directory
# Description: This directory is used for creating a lock avoiding to run
#              multiple instances of incrond.
# Default:     /var/run
#
# Example:
# lockfile_dir = /tmp


# Parameter:   lockfile_name
# Meaning:     application lock file name base
# Description: This name (appended by '.pid') is used for creating a lock
#              avoiding to run multiple instances of incrond.
# Default:     incrond
#
# Example:
# lockfile_name = incron.lock


# Parameter:   editor
# Meaning:     editor executable
# Description: This name or path is used to run as an editor for editting
#              user tables.
# Default:     vim
#
# Example:
# editor = nano

    any idea ?

    Reply Report comment

  14. Hi Nix craft,
    This is helpful I appended an email notification whenever a file is being copied but when there’s a lot of file being transferred its spamming my email because it is monitoring every event.. Any suggestion to send email per batch not per files.

    Reply Report comment

  15. i have a question,

    im running cuckoo sandbox,
    and i want to make a folder, that whenever i put a sample there, this script will perform the next action

    python submit.py

    how could i do that?

    Reply Report comment

    1. We use this line for “incrontab -e”:

      /home/user-to-be-replicated/base-directory IN_CREATE,IN_DELETE,IN_CLOSE_WRITE unison -batch -group -owner /home/user-to-be-replicated/base-directory ssh://otherhost//home/user-to-be-replicated/base-directory

      Additionally we configured password-less auth for root (private/public-key pair) and enabled incron for root:
      echo ‘root’ >> /etc/incron.allow

      I hope this helps.

      Reply Report comment

  17. I have a situation where we receive 2 or more small files of say 2k thru an FTP at the same time.
    incron only fires once for both files and passes only one file name.

    Anyone know the reason for this, and what I can do to resolve this.

    Reply Report comment

  20. This probably sounds stupid, but I’m trying to save after I enter my command. But I can’t figure out how.
    I incron -e enter my command and can’t get back out to the command line. How do I save my changes?

    Reply Report comment

Leave a Comment

Tagged as: Tags , , , , , , , , , , , , , , , , , , , ,