You can use program called cleanlinks. The cleanlinks program searches the directory tree descended from the current directory for symbolic links whose targets do not exist, and removes them. It then removes all empty directories in that directory tree. It was originally created for symbolic links based directories but works with normal directories too.

For example if you want to remove all empty directories from /tmp directory, type the command:
$ cd /tmp
$ cleanlinks

Please note that cleanlinks command is part of XFree86 project. Another method is to use combination of shell commands in script:

#/bin/bash
DIR="$1"
 [ -d $DIR ] && [ $(ls -l $DIR | wc -l) -eq 1  ] && rmdir $DIR ||  :

Save and execute a script:
$ script.sh dir1

You can also try out tmpreaper command which recursively searches for and removes files and empty directories which haven’t been accessed for a given number of seconds. Normally, it’s used to clean up directories which are used for temporary holding space, such as “/tmp”. Syntax is as follows:
tmpreaper TIME-FORMAT DIRS

Where,

• TIME-FORMAT : Defines the age threshold for removing files. The TIME-FORMAT should be a number, defaulting to hours, optionally suffixed by one character: d for days, h for hours, m for minutes, or s for seconds.
• DIRS : Directory name for example /tmp

For example, remove all files accessed 24h before:

# tmpreaper 24h /tmp

Please note that tmpreaper command is not installed by default you may need to install it using apt-get or rpm command.