≡ Menu

meta data

Parallel NFS: Read / Write Hundreds of Gigabytes Per Second

NFS is pretty old file sharing technology for UNIX based system and storage systems. However, it suffers from performance issues. NFSv4.1 address data access issues by adding a new feature called parallel NFS (pNFS) - a method of introducing Data Access Parallelism. The end result is ultra fast file sharing for clusters and high availability configurations.

The Network File System (NFS) is a stalwart component of most modern local area networks (LANs). But NFS is inadequate for the demanding input- and output-intensive applications commonly found in high-performance computing -- or, at least it was. The newest revision of the NFS standard includes Parallel NFS (pNFS), a parallelized implementation of file sharing that multiplies transfer rates by orders of magnitude.

In addition to pNFS, NFSv4.1 provides Sessions, Directory Delegation and Notifications, Multi-server Namespace, ACL/SACL/DACL, Retention Attributions, and SECINFO_NO_NAME.

Fig.01: The conceptual organization of pNFS - Image credit IBM

Fig.01: The conceptual organization of pNFS - Image credit IBM

According to wikipedia:

The NFSv4.1 protocol defines a method of separating the meta-data (names and attributes) of a filesystem from the location of the file data; it goes beyond the simple name/data separation of striping the data amongst a set of data servers. This is different from the traditional NFS server which holds the names of files and their data under the single umbrella of the server. There exists products which are multi-node NFS servers, but the participation of the client in separation of meta-data and data is limited. The NFSv4.1 client can be enabled to be a direct participant in the exact location of file data and avoid solitary interaction with the single NFS server when moving data.

The NFSv4.1 pNFS server is a collection of server resources or components; these are assumed to be controlled by the meta-data server.

The pNFS client still accesses a single meta-data server for traversal or interaction with the namespace; when the client moves data to and from the server it may be directly interacting with the set of data servers belonging to the pNFS server collection.

More information about pNFS

  1. Scale your file system with Parallel NFS
  2. Linux NFS Overview, FAQ and HOWTO Documents
  3. NFSv4 delivers seamless network access
  4. Nfsv4 Status Pages
  5. NFS article from the Wikipedia

FOSSology: Track Open Source Software Within Your Company

Hewlett-Packard has launched the FOSSology Project, a tool for tracking and monitoring the use of free and open-source software within an IT environment. The FOSSology Project is a Free Open Source Software (FOSS) project built around an open and modular architecture for analyzing software. Existing modules include license analysis, meta data extraction, and MIME type identification. This open source software tool analyzes a given set of software packages, and reports items such as the software licenses used by these packages.

More than simply reporting, "Package X uses license Y," the FOSSology tool attempts to analyze every file within the package to determine its license. The license report is thus an aggregate of all of the different licenses found to be in use by a package. A single package may be labeled as "GPL" but contain files that use other licenses (BSD, OSL, or any of the hundreds of other licenses). Even if an exact license is unknown, the license may be identifiable by common license phrases.

The availability of free open-source usage data "will help people make much better-- and affordable-- decisions about what open source to use and not use," Cote said.

=> More information available at - FOSSology Project home page.

Streamripper is cool Open Source (GPL) application that lets you record streaming mp3 to your hard drive i.e. download online streams into audio files. It can records shoutcast and icecast compatible streams. It uses meta data within a shoutcast stream to determine the beginning and end of each song, and stores the songs on your hard disk as individual mp3 files. In addition, streamripper includes a relay server for listening to the station while you are recording.

You can use it to rip (copy) streams of the following kinds:
.mp3 Shoutcast streams - the kind of streams found on shoutcast.com.
.mp3 Icecast streams
.nvs (Nullsoft Streaming Video) streams
.aac Shoutcast/Icecast streams
.ogg streams

You need to install frontend and streamripper using apt-get:
$ sudo apt-get streamripper kstreamripper

Just type kstreamripper to start kde frontend for streamripper.
$ kstreamripper &

You can also use command line application:
$ streamripper http://somehost.dom:8095/ -r

=> streamripper project.