Red Hat / CentOS Linux 4: Setup Device Mapper Multipathing

by on July 6, 2008 · 2 comments· LAST UPDATED December 3, 2008

in , ,

Multipath I/O is a fault-tolerance and performance enhancement technique whereby there is more than one physical path between the CPU in a computer system and its mass storage devices through the buses, controllers, switches, and bridge devices connecting them.

A simple example would be a SCSI disk connected to two SCSI controllers on the same computer or a disk connected to two Fibre Channel ports. Should one controller, port or switch fail, the operating system can route I/O through the remaining controller transparently to the application, with no changes visible to the applications, other than perhaps incremental latency.

This is useful for:

  1. Dynamic load balancing
  2. Traffic shaping
  3. Automatic path management
  4. Dynamic reconfiguration

Linux device-mapper

In the Linux kernel, the device-mapper serves as a generic framework to map one block device onto another. It forms the foundation of LVM2 and EVMS, software RAIDs, dm-crypt disk encryption, and offers additional features such as file-system snapshots.

Device-mapper works by processing data passed in from a virtual block device, that it itself provides, and then passing the resultant data on to another block device.

How do I setup device-mapper multipathing in CentOS / RHEL 4 update 2 or above?

Open /etc/multipath.conf file, enter:
# vi /etc/multipath.conf
Make sure following line exists and commented out:

devnode_blacklist {
        devnode "*"
}

Make sure default_path_grouping_policy option in the defaults section set to failover. Here is my sample config:

defaults {
       multipath_tool  "/sbin/multipath -v0"
       udev_dir        /dev
       polling_interval 10
       default_selector        "round-robin 0"
       default_path_grouping_policy    failover
       default_getuid_callout  "/sbin/scsi_id -g -u -s /block/%n"
       default_prio_callout    "/bin/true"
       default_features        "0"
       rr_min_io              100
       failback                immediate
}

Save and close the file. Type the following command to load drivers:
# modprobe dm-multipath
# modprobe dm-round-robin

Start the service, enter:
# /etc/init.dmultipathd start
multipath is used to detect multiple paths to devices for fail-over or performance reasons and coalesces them:
# multipath -v2
Turn on service:
# /sbin/chkconfig multipathd on
Finally, create device maps from partition tables:
# kpartx -a /dev/mapper/mpath#
You need to use fdisk on the underlying disks such as /dev/sdc.

References:

  • man page kpartx,multipath, udev, dmsetup and hotplug
TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 2 comments… read them below or add one }

1 pd July 1, 2009 at 9:58 am

i am trying to setup multipath on centos 5 and using SAN HP storage disks
problem is /dev/dm-5, /dev/dm-6 and /dev/dm-7 automatically coming after reboot.
i have added three multipath in multipath.conf. i am not able to mount these device /dev/dm-2, /dev/dm-3 and /dev/dm4

why these new device maps automatically added??
[root@BL480 ~]# multipath -l
extravm (360014380024d224a0001000000420000) dm-4 HP,HSV300
[size=60G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:3 sdc 8:32 [active][undef]
\_ 0:0:1:3 sdf 8:80 [active][undef]
joomla (360014380024d224a00010000003c0000) dm-3 HP,HSV300
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:2 sdb 8:16 [active][undef]
\_ 0:0:1:2 sde 8:64 [active][undef]
sqlvm (360014380024d224a0001000000140000) dm-2 HP,HSV300
[size=80G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:1 sda 8:0 [active][undef]
\_ 0:0:1:1 sdd 8:48 [active][undef]
[root@BL480 ~]# fdisk -l

Disk /dev/cciss/c0d0: 36.3 GB, 36380413440 bytes
64 heads, 32 sectors/track, 34695 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/cciss/c0d0p1 * 1 100 102384 83 Linux
/dev/cciss/c0d0p2 101 34695 35425280 8e Linux LVM

Disk /dev/sda: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 1 10443 83883366 83 Linux

Disk /dev/sdb: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 13055 104857568+ 83 Linux

Disk /dev/sdc: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 61440 62914544 83 Linux

Disk /dev/sdd: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdd1 1 10443 83883366 83 Linux

Disk /dev/sde: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sde1 1 13055 104857568+ 83 Linux

Disk /dev/sdf: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/sdf1 1 61440 62914544 83 Linux

Disk /dev/dm-2: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/dm-2p1 1 10443 83883366 83 Linux

Disk /dev/dm-3: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/dm-3p1 1 13055 104857568+ 83 Linux

Disk /dev/dm-4: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/dm-4p1 1 61440 62914544 83 Linux

Disk /dev/dm-5: 85.8 GB, 85896566784 bytes
255 heads, 63 sectors/track, 10442 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-5 doesn’t contain a valid partition table

Disk /dev/dm-6: 107.3 GB, 107374150144 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-6 doesn’t contain a valid partition table

Disk /dev/dm-7: 64.4 GB, 64424493056 bytes
255 heads, 63 sectors/track, 7832 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-7 doesn’t contain a valid partition table

Reply

2 Charanjit Singh February 19, 2010 at 6:14 am

Thanks for this article. But there is a slightly mistake in line “# /etc/init.dmultipathd start” this should be amend as “# /etc/init.d/multipathd start”.

Regards
Charanjit Singh Cheema

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , , , , , ,

Previous post:

Next post: