Linux User does not exist – weird problem and solution

Recently I received a support request as follows:
After adding a user using useradd command:
useradd -s /bin/false -d foo
passwd foo

Following error reported
user foo does not exist

However, after sometime user foo allowed to login. What is going on? Is my system hacked?

First I went through system password database located at /etc/passwd, /etc/shadow, /etc/group - there exists an entry for foo user.

After close investigation I found name service cache daemon (nscd). It is a daemon that provides a cache for the most common name service requests including caching of /etc/passwd file (thanks for top and lsof command for hint).

So the solution is to stop the nscd service add user and start nscd again. This time it worked.
# /etc/init.d/nscd stop
# useradd -s /bin/false -d bar
# passwd bar
# /etc/init.d/nscd start


Now bar is allowed to login immediately :).

Refer to nscd man page for detailed information (hint setup correct TTL to get rid of this problem and upgrade to latest version):
man nscd
man nscd.conf

Featured Articles:

• 20 Linux System Monitoring Tools Every SysAdmin Should Know
• 20 Linux Server Hardening Security Tips
• Linux: 20 Iptables Examples For New SysAdmins
• My 10 UNIX Command Line Mistakes
• 25 PHP Security Best Practices For Sys Admins
• The Novice Guide To Buying A Linux Laptop
• Top 5 Email Client For Linux, Mac OS X, and Windows Users
• Top 20 OpenSSH Server Best Security Practices
• Top 10 Open Source Web-Based Project Management Software