NFS is pretty old file sharing technology for UNIX based system and storage systems. However, it suffers from performance issues. NFSv4.1 address data access issues by adding a new feature called parallel NFS (pNFS) - a method of introducing Data Access Parallelism. The end result is ultra fast file sharing for clusters and high availability configurations.
The Network File System (NFS) is a stalwart component of most modern local area networks (LANs). But NFS is inadequate for the demanding input- and output-intensive applications commonly found in high-performance computing -- or, at least it was. The newest revision of the NFS standard includes Parallel NFS (pNFS), a parallelized implementation of file sharing that multiplies transfer rates by orders of magnitude.
In addition to pNFS, NFSv4.1 provides Sessions, Directory Delegation and Notifications, Multi-server Namespace, ACL/SACL/DACL, Retention Attributions, and SECINFO_NO_NAME.
Fig.01: The conceptual organization of pNFS - Image credit IBM
According to wikipedia:
The NFSv4.1 protocol defines a method of separating the meta-data (names and attributes) of a filesystem from the location of the file data; it goes beyond the simple name/data separation of striping the data amongst a set of data servers. This is different from the traditional NFS server which holds the names of files and their data under the single umbrella of the server. There exists products which are multi-node NFS servers, but the participation of the client in separation of meta-data and data is limited. The NFSv4.1 client can be enabled to be a direct participant in the exact location of file data and avoid solitary interaction with the single NFS server when moving data.
The NFSv4.1 pNFS server is a collection of server resources or components; these are assumed to be controlled by the meta-data server.
The pNFS client still accesses a single meta-data server for traversal or interaction with the namespace; when the client moves data to and from the server it may be directly interacting with the set of data servers belonging to the pNFS server collection.
More information about pNFS
- Scale your file system with Parallel NFS
- Linux NFS Overview, FAQ and HOWTO Documents
- NFSv4 delivers seamless network access
- Nfsv4 Status Pages
- NFS article from the Wikipedia
A few days ago I noticed that NFS performance between a web server node and NFS server went down by 50%. NFS was optimized and the only thing was updated Red Hat kernel v5.2. I also noticed same trend on CentOS 5.2 64 bit edition.
NFS server crashed each and every time web server node tried to store a large file 20-100 MB each. Read performance was fine but write performance went to hell. Finally, I had to rollback the updates. Recently, while reading Red Hat site I came across the solution.
Updated kernel packages that fix various security issues and several bugs are now available for Red Hat Enterprise Linux 5:
* a 50-75% drop in NFS server rewrite performance, compared to Red Hat
Enterprise Linux 4.6, has been resolved.
After upgrading kernel on both server and client my issue resolved:
# yum update
Recently, I noticed something strange about Apache 2.2.3 version running on CentOS Linux 5 64 bit version. We have centralized NFS server and all 3 web server load balanced using hardware front end (another box running LVS).
All Apache server picks up file via NFS i.e DocumentRoot is set over NFS. The small file such as 2 MB or 5 MB get downloaded correctly but large size files failed to download. Another problem was some clients reported that the file get download but cannot open due to file corruption issue.
After investigation and a little bit googling I came across the solution. You need to disable following two options:
- EnableMMAP - This directive controls whether the httpd may use memory-mapping if it needs to read the contents of a file during delivery. By default, when the handling of a request requires access to the data within a file -- for example, when delivering a server-parsed file using mod_include -- Apache memory-maps the file if the OS supports it.
- EnableSendfile - This directive controls whether httpd may use the sendfile support from the kernel to transmit file contents to the client. By default, when the handling of a request requires no access to the data within a file -- for example, when delivering a static file -- Apache uses sendfile to deliver the file contents without ever reading the file if the OS supports it.
However, these two directives are known to have problem with a network-mounted DocumentRoot (e.g., NFS or SMB), the kernel may be unable to serve the network file through its own cache. So just open httpd.conf on all boxes and changes the following:
Just restart the web server and voila!
# service httpd restart
NFS (Network File System) client allows you to access shared directory from Linux client. The computer sharing the directory is called the NFS server (it can be NAS server too) and the computers or devices connecting to that server are called clients. The clients need to use the mount command to access the shared directory.
[click to continue…]