CentOS / Redhat: Setup NFS v4.0 File Server

by on June 2, 2010 · 16 comments· LAST UPDATED June 4, 2012

in , ,

How do I setup NFS v4.0 distributed file system access server under CentOS / RHEL v5.x for sharing files with UNIX and Linux workstations? How to export a directory with NFSv4? How to mount a directory with NFSv4?

Network File System (NFS) is a network file system protocol originally developed by Sun Microsystems. It allows your users or client compute to access files over a network. Linux and UNIX like operating systems (including MS-Windows) can mount file system over a network and work as they are mounted locally. This is perfect for sharing files or centralized home directories.

NFS version 4 provides the following benefits over NFSv3 or earlier NFS versions:

  1. Performance improvements
  2. Mandates security and ACL
  3. NFS v4 by default works over TCP s
  4. Easy to setup firewall option
  5. And much more.

Required Packages

You need to install the following packages:

  • nfs-utils - The nfs-utils package provides a daemon for the kernel NFS server and related tools, which provides a much higher level of performance than the traditional Linux NFS server used by most users.
  • portmap - The portmap package should be installed on any machine which acts as a server for protocols using RPC.
  • nfs4-acl-tools - This package contains commandline and GUI ACL utilities for the Linux NFSv4 client.

Install NFS Server

Type the following command (install nfs4-acl-tools and nfs-utils on client systems too):
# yum install nfs-utils nfs4-acl-tools portmap
Sample outputs:

Loaded plugins: downloadonly, protectbase, rhnplugin, security, verify
0 packages excluded due to repository protections
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package nfs-utils.x86_64 1:1.0.9-44.el5 set to be updated
---> Package nfs4-acl-tools.x86_64 0:0.3.3-1.el5 set to be updated
---> Package portmap.x86_64 0:4.0-65.2.2.1 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
 Package           Arch      Version              Repository               Size
================================================================================
Installing:
 nfs-utils         x86_64    1:1.0.9-44.el5       rhel-x86_64-server-5    390 k
 nfs4-acl-tools    x86_64    0.3.3-1.el5          rhel-x86_64-server-5     44 k
 portmap           x86_64    4.0-65.2.2.1         rhel-x86_64-server-5     38 k
Transaction Summary
================================================================================
Install       3 Package(s)
Upgrade       0 Package(s)
Total download size: 472 k
Is this ok [y/N]: y
Downloading Packages:
(1/3): portmap-4.0-65.2.2.1.x86_64.rpm                   |  38 kB     00:00
(2/3): nfs4-acl-tools-0.3.3-1.el5.x86_64.rpm             |  44 kB     00:00
(3/3): nfs-utils-1.0.9-44.el5.x86_64.rpm                 | 390 kB     00:00
--------------------------------------------------------------------------------
Total                                           1.2 MB/s | 472 kB     00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : portmap                                                  1/3
  Installing     : nfs4-acl-tools                                           2/3
  Installing     : nfs-utils                                                3/3
Installed:
  nfs-utils.x86_64 1:1.0.9-44.el5      nfs4-acl-tools.x86_64 0:0.3.3-1.el5
  portmap.x86_64 0:4.0-65.2.2.1
Complete!

Share File System

/etc/exports This is main NFS server config file which controls what directories the NFS server exports (shared with client). It use the following format:

/directory1 server.example.com(options)
/directory2 192.168.1.0/24(options)
/directory3 192.168.1.5(options) 192.168.1.15(options) pc202.nixcraft.net.in(options) 

You can share /sales file system as follows. Edit /etc/exports, enter:
# vi /etc/exports
Add configuration as follows:

/sales                       192.168.1.15(rw,sync,fsid=0) 192.168.1.16(rw,sync,fsid=0)

Where,
/sales - Share this directory.

  1. 192.1681.15 and 192.168.1.16 - Users from 192.168.1.15 and 192.168.1.16 are allowed to mount /sales with the read-write permissions.
  2. rw - Read write option.
  3. fsid=0 - Export a directory over NFS v4. NFSv4 has a concept of a root of the overall exported filesystem. The export point exported with fsid=0 will be used as this root. The /sales directory will be root for clients. For example, if you got /sales/mumbai, /sales/pune subdir, then client would see them as /mumbai and /pune directory. Please note that this can only export one directory with the fsid=0 option.

Save and close the file. Turn on services:
# chkconfig nfs on
# chkconfig portmap on

Start both portmap and nfs services, enter:
# service portmap start <-- for NFSv3 support
# service nfs start

Sample outputs:

Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]

Please note that portmap service is not required for NFSv4.

Optional: NFS Server Configuration GUI Tool

Type the following command to use GUI tool:
# system-config-nfs
Sample outputs:

Fig.01: Linux NFS Server Configuration

Fig.01: Linux NFS Server Configuration

NFSv4 Firewall Configuration

Edit /etc/sysconfig/iptables, enter:
# vi /etc/sysconfig/iptables
Open TCP port # 2049 which is used by NFSv4. Add the following lines, ensuring that they appear before the final LOG and DROP lines for the RH-Firewall-1-INPUT chain:

-A RH-Firewall-1-INPUT -s 192.168.1.0/24 -m state --state NEW -p tcp --dport 2049 -j ACCEPT

Save and close the file. Restart RHEL/CentOS firewall:
# service iptables restart
See how to configure firewall for NFSv3 and earlier under RHEL / CentOS Linux.

TCP Wrapper Configuration

TCP Wrapper is a host-based networking ACL system, used to filter network access to Internet. Edit /etc/hosts.deny, enter:
# vi /etc/hosts.deny
Add the following lines (useful for both NFSv4 and NFSv3):
portmap:ALL
Finally, edit /etc/hosts.allow and add your subnet:
portmap:192.168.1.0/24
Save and close the file.

NFS Client Configuration

The clients can then mount the NFSv4 export using the following command:
# mkdir /sales
# mount -t nfs4 servername:/ /sales/
# df -H
# su - username
$ cd /sales/mumbai
$ ls
$ >testfile
$ ls testfile && rm testfile

A Note About User Management

Use NIS or OpenLDAP for user management for large number of users. If you've small number of NFS clients add them to your systems using the useradd command. Make sure UID and GID matches correctly. For example, if user vivek (UID=500) is part of group vivek (gid=500) and sales group (Gid=502) on NFSv4 server, than use the following command to add user to NFSv4 client:
# grep -q '^sales' /etc/group || /usr/sbin/groupadd -g 502 sales
# /usr/sbin/useradd -s /bin/bash -d /sales -M -u 500 -g 500 -G 502 sales
# su - sales
$ pwd
$ ls && cd mumbai && >testfile && ls -l testfile && rm testfile

The above command matches client and server UIDs and GIDs. Otherwise you will get permission denied message on NFSv4 clients. As I said earlier, for a large number of NFSv4 users/clients, use centralized authentication systems such as NIS or OpenLDAP.

Mounting NFS File Systems Using /etc/fstab

Edit /etc/fstab, enter:
# vi /etc/fstab
Append the entry, enter:

server:/    /sales  nfs4   soft,intr,rsize=8192,wsize=8192,nosuid

Save and close the file. Make sure netfs service is turned on:
# chkconfig netfs on

How Do I See NFS Statistics?

To displays statistics kept about NFS client and server activity, enter:
# nfsstat
Sample outputs:

Server rpc stats:
calls      badcalls   badauth    badclnt    xdrcall
28131      0          0          0          0
Server nfs v3:
null         getattr      setattr      lookup       access       readlink
10        0% 12302    58% 62        0% 166       0% 2122     10% 35        0%
read         write        create       mkdir        symlink      mknod
7         0% 4039     19% 52        0% 3         0% 0         0% 0         0%
remove       rmdir        rename       link         readdir      readdirplus
47        0% 2         0% 6         0% 0         0% 1         0% 2273     10%
fsstat       fsinfo       pathconf     commit
21        0% 13        0% 0         0% 4         0%
Server nfs v4:
null         compound
8         0% 6726     99%
Server nfs v4 operations:
op0-unused   op1-unused   op2-future   access       close        commit
0         0% 0         0% 0         0% 54        0% 2019      7% 0         0%
create       delegpurge   delegreturn  getattr      getfh        link
0         0% 0         0% 1         0% 8563     30% 2094      7% 0         0%
lock         lockt        locku        lookup       lookup_root  nverify
0         0% 0         0% 0         0% 78        0% 0         0% 0         0%
open         openattr     open_conf    open_dgrd    putfh        putpubfh
2022      7% 0         0% 14        0% 0         0% 6710     24% 0         0%
putrootfh    read         readdir      readlink     remove       rename
12        0% 70        0% 16        0% 7         0% 5         0% 3         0%
renew        restorefh    savefh       secinfo      setattr      setcltid
1         0% 2022      7% 2025      7% 0         0% 4         0% 5         0%
setcltidconf verify       write        rellockowner
5         0% 0         0% 2003      7% 0         0%

How Do I Display Information About Shared Directories?

To see mount information for an NFS server (rpc portmap service is required), enter:
# showmount -e
# showmount -d
# showmount -a server.ip

A Note About NFSv4 Services

  1. The NFSv4 server works without the portmap, rpc.lockd, and rpc.statd daemons. The rpc.mountd daemon is still required on the server.
  2. The NFSv4 client works without rpc.lockd and rpc.statd.
  3. However, if you are going to mix NFSv4 and NFSv3 than make sure you start above services on both client and server.

Recommend readings:

For more information on use of the nfs server, client and additional options, please refer to the following man pages:
man nfs
man 5 exports
man 8 mount
man 8 umount
man 8 nfsstat
man showmount

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 16 comments… read them below or add one }

1 jaysunn June 2, 2010 at 10:29 pm

Hey,
Nice work as always. I have shared a hundred shares via NFS in my time however I did not use some of the above options.

for i in $(cat awesome_tutorial.html) ; do
echo $i

done

jaysunn

Reply

2 Solaris June 3, 2010 at 12:26 pm

Hey Vivek,

I don’t know for sure but NFS v4 ACL are not POSIX compatible, they are more windows oriented!

Also there is a mapping file that you can use if you don’t want your uid,gid’s to be synchronized on both systems but you are limited because you can’t mount a user with more remote gids and the case a user belongs to multiple groups on the remote system is often.

If I am wrong please correct me :)

Reply

3 Anwar June 3, 2010 at 1:29 pm

Well, define Process is quite well but something is missing there that is
nfsv4 incude these Daemons for securing the Data accross the network

1> gssd (creates security context on RPC client for exchanging RPC information using RPCSEC)
2> idmapd (Maps local users and group names to NFSv4 ids
3> svcgssd (Creates Security context on RPC server for exchanging RPC information using SecureRPC )

Reply

4 sai March 18, 2011 at 1:01 pm

can u explain what type service need for nfsv4 expect this nfs,nfslock,rpcbind
and also some step to configure nfsv4 thanks

Reply

5 Jace June 3, 2010 at 5:31 pm

I have used Samba and configured Samba on CentOS 5, and use that on private networks. What advantages does NFS offer over Samba? What are the pro’s and cons of each one and what applications does each one have and each one suit better?

Jace

Reply

6 ANders June 10, 2010 at 6:31 pm

You get speed with NFS compared to Samba.
And NFSv4 support Kerberos, so you get security too. You have Kerberos in MS Windows AD too.

Reply

7 Planet_Bob June 3, 2010 at 10:39 pm

Nice info !
Short sweet and to the point !
Keep up the good work !

Might want to list nfswatch util…
http://nfswatch.sourceforge.net/

Reply

8 Planet_Bob June 3, 2010 at 10:42 pm

Also might be useful…link to SUN/Oracle’s NFS tuning site
Analyzing NFS Performance:
http://docs.sun.com/app/docs/doc/805-4448/6j47cnj0i?l=en&a=view

Reply

9 PieterT June 30, 2010 at 2:15 pm

I cannot get it to work. I am not able to mount a directory on a client from the server.

[root@ph]~# mount /mnt/wrapper/
mount: mount to NFS server ’10.x.w.z’ failed: RPC Error: Program not registered.

When I try to start netfs:
[root@ph]~# /etc/init.d/netfs start
Mounting NFS filesystems: mount: mount to NFS server ’10.x.w.z’ failed: RPC Error: Program not registered.
[FAILED]
Mounting other filesystems: [ OK ]

What am I missing?

Reply

10 knn sarma February 15, 2011 at 9:15 am

awesome.thanku

Reply

11 dhananjay k sharma April 16, 2011 at 6:45 am

good article….

Reply

12 Rob August 7, 2011 at 3:36 am

May need update for Fedora 15. It doesn’t use portmap – it uses rpcbind.

Reply

13 Yeswanth Reddy March 30, 2012 at 2:02 pm

Unable to mount a file system

There are two common errors that mount produces when it is unable to mount a volume. These are:

failed, reason given by server: Permission denied

This means that the server does not recognize that you have access to the volume.

Check your /etc/exports file and make sure that the volume is exported and that your client has the right kind of access to it. For example, if a client only has read access then you have to mount the volume with the ro option rather than the rw option.

Make sure that you have told NFS to register any changes you made to /etc/exports since starting nfsd by running the exportfs command. Be sure to type exportfs -ra to be extra certain that the exports are being re-read.

Check the file /proc/fs/nfs/exports and make sure the volume and client are listed correctly. (You can also look at the file /var/lib/nfs/xtab for an unabridged list of how all the active export options are set.) If they are not, then you have not re-exported properly. If they are listed, make sure the server recognizes your client as being the machine you think it is. For example, you may have an old listing for the client in /etc/hosts that is throwing off the server, or you may not have listed the client’s complete address and it may be resolving to a machine in a different domain. One trick is login to the server from the client via ssh or telnet; if you then type who, one of the listings should be your login session and the name of your client machine as the server sees it. Try using this machine name in your /etc/exports entry. Finally, try to ping the client from the server, and try to ping the server from the client. If this doesn’t work, or if there is packet loss, you may have lower-level network problems.

It is not possible to export both a directory and its child (for example both /usr and /usr/local). You should export the parent directory with the necessary permissions, and all of its subdirectories can then be mounted with those same permissions.

RPC: Program Not Registered: (or another “RPC” error):

This means that the client does not detect NFS running on the server. This could be for several reasons.

First, check that NFS actually is running on the server by typing rpcinfo -p on the server. You should see something like this:

program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100011 1 udp 749 rquotad
100011 2 udp 749 rquotad
100005 1 udp 759 mountd
100005 1 tcp 761 mountd
100005 2 udp 764 mountd
100005 2 tcp 766 mountd
100005 3 udp 769 mountd
100005 3 tcp 771 mountd
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
300019 1 tcp 830 amd
300019 1 udp 831 amd
100024 1 udp 944 status
100024 1 tcp 946 status
100021 1 udp 1042 nlockmgr
100021 3 udp 1042 nlockmgr
100021 4 udp 1042 nlockmgr
100021 1 tcp 1629 nlockmgr
100021 3 tcp 1629 nlockmgr
100021 4 tcp 1629 nlockmgr

This says that we have NFS versions 2 and 3, rpc.statd version 1, network lock manager (the service name for rpc.lockd) versions 1, 3, and 4. There are also different service listings depending on whether NFS is travelling over TCP or UDP. UDP is usually (but not always) the default unless TCP is explicitly requested.

Reply

14 siva October 11, 2012 at 4:07 am

In my view this procedure has its own bottleneck. If you want to mount multiple directory then this procedure is invalid.

http://www.crazysquirrel.com/computing/debian/servers/setting-up-nfs4.jspx

The above URL explain clarify those doubt in related to nfs4.

Reply

15 Yossi April 29, 2013 at 2:59 pm

I have configured NFS server and Client.
Permission is (rw)
I can get file from Server to Client.
But I can’t save file from Client to server.
I am using NFSv4 in both Client and server.
in my iptables:
-A RH-Firewall-1-INPUT -s 10.133.9.106/30 -m state –state NEW -p tcp –dport 2049 -j ACCEPT

is the Client address.

Reply

16 mandela900 February 9, 2014 at 2:59 pm

why are you using rsize=8192 and wsize=8192 for nfs4?, i think this is the default for nfs2

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: