Linux incrond inotify Monitor Directories For Changes And Take Action

last updated in Categories , , , ,

I want to copy (rsync to a remote server) a directory tree whenever file uploaded or deleted in /var/www/html/upload/ directory under Linux operating systems for backup purpose and/or load balancing purpose without getting into complicated file sharing setups such as NFS or GFS OR iSCSI storage. How do I monitor /var/www/html/upload/ and its subdirectory for new files and executes rsync command to make copy back to www2.example.com:/var/www/html/upload/?

inotify is an inode-based filesystem notification technology. It provides the possibility to monitor various events on files in filesystems. It is a very much great replacement of (obsolete) dnotify. inotify brings a comfortable way how to manage files used in your applications. The incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to Linux or Unix cron jobs. This page shows how to set up incrond and sync files between two web server nodes.

How to install incrond

Type the following dnf command/yum command under RHEL / Fedora / CentOS Linux:
$ sudo yum install incron
Type the apt command/apt-get command under Debian / Ubuntu Linux:
$ sudo apt-get install incron
Linux install incrond - inotify cron (incron) daemon

Configuration Files

  • /etc/incron.conf – Main incron configuration file
  • /etc/incron.d/ – This directory is examined by incrond for system table files. You should put all your config file here as per directory or domain names.
  • /etc/incron.allow – This file contains users allowed to use incron.
  • /etc/incron.deny – This file contains users denied to use incron.
  • /var/spool/incron – This directory is examined by incrond for user table files which is set by users running the incrontab command.

incron Syntax

The syntax is as follows:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload

Where,

  • <directory> – It is nothing but path which is an absolute filesystem path such as /home/data. Any changes made to this path will result into command or action.
  • <file change mask> – Mask is is nothing but various file system events such as deleting a file. Each event can result into command execution. Use the following masks:
    • IN_ACCESS – File was accessed (read)
    • IN_ATTRIB – Metadata changed (permissions, timestamps, extended attributes, etc.)
    • IN_CLOSE_WRITE – File opened for writing was closed
    • IN_CLOSE_NOWRITE – File not opened for writing was closed
    • IN_CREATE – File/directory created in watched directory
    • IN_DELETE – File/directory deleted from watched directory
    • IN_DELETE_SELF – Watched file/directory was itself deleted
    • IN_MODIFY – File was modified
    • IN_MOVE_SELF – Watched file/directory was itself moved
    • IN_MOVED_FROM – File moved out of watched directory
    • IN_MOVED_TO – File moved into watched directory
    • IN_OPEN – File was opened
    • The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above events.
  • <command or action> – Run command or scripts when mask matched on given directory.
  • options – It can be any one of the following with command (i.e. you can pass it as arg to your command):
    1. $$ – dollar sign
    2. $@ – watched filesystem path (see above)
    3. $# – event-related file name
    4. $% – event flags (textually)
    5. $& – event flags (numerically)

Turn Service On

Type the following command on a CentOS/RHEL:
# service incrond start
# chkconfig incrond on
###################################################
### systemd based Linux distro such as CentOS/RHEL 7.x/8.x, try ##
###################################################
# systemctl enable incrond.service
# systemctl start incrond.service

Linux incrond inotify monitor dir for changes and take action

Type the following command to edit your incrontab
incrontab -e
Run logger command when file created or deleted from /tmp directory:
/tmp IN_ALL_EVENTS logger "/tmp action for $# file"
Save and close the file. Now cd to /tmp and create a file:
$ cd /tmp
$ >foo
$ rm foo
To see message, enter:
$ sudo tail -f /var/log/messages
Sample outputs:
Jul 17 18:39:25 vivek-desktop logger: "/tmp action for foo file"

How Do I Run Rsync Command To Replicate Files For /var/www/html/upload Directory?

Type the following command:
# incrontab -e
Append the following command:
/var/www/html/upload/ IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
Now, wherever files are uploaded in /var/www/html/upload/ directory, rsync will be executed to sync files to www2.example.com server. Make sure ssh keys are set for password less login.

How Do I Monitor /var/www/html/upload/ and Its Subdirectories Recursively?

You cannot monitor /var/www/html/upload/ directory recursively. However, you can use the find command to add all sub-directories as follows:
find /var/www/html/upload -type d -print0 | xargs -0 -I{} echo "{} IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/" > /etc/incron.d/webroot.conf
This will create /etc/incron.d/webroot.conf config as follows:

/var/www/html/upload IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/css IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/1 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/js IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/3 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/11 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2010/12 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/2 IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/files IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/
/var/www/html/upload/images IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /usr/bin/rsync --exclude '*.tmp' -a /var/www/html/upload/ user@www2.example.com:/var/www/html/upload/

How Do I Troubleshoot Problems?

You need to see /var/log/cron log file:
# tail -f /var/log/cron
# grep something /var/log/cron

Further readings:

  • inotify project home page here
  • man pages – incrontab(5), incrond, and incron.conf

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Start the discussion at www.nixcraft.com

Historical Comment Archive

30 comment

  1. Awesome man! I liked the most about rndc reload. Now no need to keep running rndc reload every hour or so.

      1. No I don’t. Just gave a thought. I know XName.org reloads it every hour. I don’t have too many zones.

  2. None of my machines using the default CentOS repos seem to have incron available to install. Which repos are you using? My machines are Centos 5.5 ones (one 32-bit, one 64-bit).

    1. It is from Fedora Project – EPEL repo. Install epel repo and you will get access to tons of binary packages.

      rpm -qi incron | grep -i vendor
Version     : 0.5.5                             Vendor: Fedora Project

      HTH

      1. After adding EPEL per the tutorial you linked, I was able to add incron onto my machine. Thanks for the quick response and very useful information you’ve provided.

  3. Silly, this can be done with a simple 4 or 5 line script in python.

    hints
    os.path.exists
    os.unlink
    popen

    1. Ya sure, but you need to run it as a daemon, it would be slow/cpuhog. incron is in C, so works much faster. There may be some python-inotify library which can be used instead of the above suggestion.
      Also, using this method, you could write a bash daemon! (I have one on my local system to check that if pidgin runs, ensure skype runs, if pidgin doesn’t kill skype).

    2. Try that on tons of thousandths of files and you will see that your Python will perform in tens of seconds rather than milliseconds. You will say that there is no big difference but this will translate in disk reads a the physical level rather than operating system level triggers.

      I speak from experience. I am not using incrond but something similar in production and it performs way faster that the similar Python code.

      Use the right tool for the job.

      1. That’s one of the libraries, there’s a C extension also available which directly handles with the kernel, i.e. responds to kernel events. Just google it out.

  5. From what I’m seeing online, the FreeBSD and Mac OS X implementation is called kqueue rather than inotify.

    1. Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

      Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

      (If this ends up double posting, apologies, but it didn’t appear to post or even be in moderation after I initially submitted it so re-adding it.)

      1. DRBD is much better solution if you need to mirror entire partition in real time. It is like network based RAID-1.

  6. Something else I wanted to bring up about using inotify after having looked more into it. There’s a handy option lsyncd that will work better than incrond for rsyncing directories. The project’s home page is at http://code.google.com/p/lsyncd/

    Is there a consideration of bring up lsync for either this tutorial or another one? It appears to possibly work better due to handling recursion in a directory for rsyncing using inotify based on what I’m seeing.

  7. Hi all,

    Is it possible to set incrontab in bash script? I tried to set normal crontab and it works without any problem but not “incrontab” :(

    Any idea?

    R

  8. This is easily the best website to check for new ideas. I was struggling with inotify for this kind of job and I wouldn’t have find this incron tool by myself.

  9. Thanks for the tutorial. I was exactly looking for a tool like this one – thanks to this blog it even comes with a complete tutorial! :)

  11. HI Nix craft,

    when i use incrontab out of the box, i mean no changes,

    i have no result, as if the cmd was not executed.

    my incron line is :

    /crrae IN_CREATE,IN_CLOSE_WRITE,IN_MOVED_TO /usr/bin/unix2dos $#

    the log give information as if everything was ok but no actually

    when i change my incron to :

    /crrae IN_CREATE,IN_CLOSE_WRITE,IN_MOVED_TO /usr/bin/unix2dos /crrae/$#

    the log give me this:

    May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpPbqoFd)
May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpN2Odxf)
May  4 18:15:10 srvovs incrond[20859]: (root) CMD (/usr/bin/unix2dos /crrae/u2dtmpN2Odxf)

    multiple times

    here is the incron.conf:

    [root@srvovs ~]# cat /etc/incron.conf
#
# *** incron example configuration file ***
#
# (c) Lukas Jelinek, 2007, 2008
#


# Parameter:   system_table_dir
# Meaning:     system table directory
# Description: This directory is examined by incrond for system table files.
# Default:     /etc/incron.d
#
# Example:
# system_table_dir = /var/spool/incron.systables


# Parameter:   user_table_dir
# Meaning:     user table directory
# Description: This directory is examined by incrond for user table files.
# Default:     /var/spool/incron
#
# Example:
# user_table_dir = /var/spool/incron.usertables


# Parameter:   allowed_users
# Meaning:     allowed users list file
# Description: This file contains users allowed to use incron.
# Default:     /etc/incron.allow
#
# Example:
# allowed_users = /etc/incron/allow


# Parameter:   denied_users
# Meaning:     denied users list file
# Description: This file contains users denied to use incron.
# Default:     /etc/incron.deny
#
# Example:
# denied_users = /etc/incron/deny


# Parameter:   lockfile_dir
# Meaning:     application lock file directory
# Description: This directory is used for creating a lock avoiding to run
#              multiple instances of incrond.
# Default:     /var/run
#
# Example:
# lockfile_dir = /tmp


# Parameter:   lockfile_name
# Meaning:     application lock file name base
# Description: This name (appended by '.pid') is used for creating a lock
#              avoiding to run multiple instances of incrond.
# Default:     incrond
#
# Example:
# lockfile_name = incron.lock


# Parameter:   editor
# Meaning:     editor executable
# Description: This name or path is used to run as an editor for editting
#              user tables.
# Default:     vim
#
# Example:
# editor = nano

    any idea ?

  12. Hi Nix craft,
    This is helpful I appended an email notification whenever a file is being copied but when there’s a lot of file being transferred its spamming my email because it is monitoring every event.. Any suggestion to send email per batch not per files.

  13. i have a question,

    im running cuckoo sandbox,
    and i want to make a folder, that whenever i put a sample there, this script will perform the next action

    python submit.py

    how could i do that?

  14. Has anyone got incron & unison successfully working together to sync remote servers, all on CentOS 6? If so, can you explain, please?

    Thanks

    1. We use this line for “incrontab -e”:

      /home/user-to-be-replicated/base-directory IN_CREATE,IN_DELETE,IN_CLOSE_WRITE unison -batch -group -owner /home/user-to-be-replicated/base-directory ssh://otherhost//home/user-to-be-replicated/base-directory

      Additionally we configured password-less auth for root (private/public-key pair) and enabled incron for root:
      echo ‘root’ >> /etc/incron.allow

      I hope this helps.

  15. I have a situation where we receive 2 or more small files of say 2k thru an FTP at the same time.
    incron only fires once for both files and passes only one file name.

    Anyone know the reason for this, and what I can do to resolve this.

  16. Awesome, exactly what I needed. I use it with cups and avahi to print PDFs on my iPhone and automatically email them to myself. Many thanks.

  18. This probably sounds stupid, but I’m trying to save after I enter my command. But I can’t figure out how.
    I incron -e enter my command and can’t get back out to the command line. How do I save my changes?

    Still, have a question? Get help on our forum!

Tagged as: Tags , , , , , , , , , , , , , , , , , , ,