Multipath I/O is a fault-tolerance and performance enhancement technique whereby there is more than one physical path between the CPU in a computer system and its mass storage devices through the buses, controllers, switches, and bridge devices connecting them.
A simple example would be a SCSI disk connected to two SCSI controllers on the same computer or a disk connected to two Fibre Channel ports. Should one controller, port or switch fail, the operating system can route I/O through the remaining controller transparently to the application, with no changes visible to the applications, other than perhaps incremental latency.
This is useful for:
- Dynamic load balancing
- Traffic shaping
- Automatic path management
- Dynamic reconfiguration
Linux device-mapper
In the Linux kernel, the device-mapper serves as a generic framework to map one block device onto another. It forms the foundation of LVM2 and EVMS, software RAIDs, dm-crypt disk encryption, and offers additional features such as file-system snapshots.
Device-mapper works by processing data passed in from a virtual block device, that it itself provides, and then passing the resultant data on to another block device.
How do I setup device-mapper multipathing in CentOS / RHEL 4 update 2 or above?
Open /etc/multipath.conf file, enter:
# vi /etc/multipath.conf
Make sure following line exists and commented out:
devnode_blacklist {
devnode "*"
}Make sure default_path_grouping_policy option in the defaults section set to failover. Here is my sample config:
defaults {
multipath_tool "/sbin/multipath -v0"
udev_dir /dev
polling_interval 10
default_selector "round-robin 0"
default_path_grouping_policy failover
default_getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
default_prio_callout "/bin/true"
default_features "0"
rr_min_io 100
failback immediate
}
Save and close the file. Type the following command to load drivers:
# modprobe dm-multipath
# modprobe dm-round-robin
Start the service, enter:
# /etc/init.dmultipathd start
multipath is used to detect multiple paths to devices for fail-over or performance reasons and coalesces them:
# multipath -v2
Turn on service:
# /sbin/chkconfig multipathd on
Finally, create device maps from partition tables:
# kpartx -a /dev/mapper/mpath#
You need to use fdisk on the underlying disks such as /dev/sdc.
References:
- man page kpartx,multipath, udev, dmsetup and hotplug
Featured Articles:
- 20 Linux System Monitoring Tools Every SysAdmin Should Know
- 20 Linux Server Hardening Security Tips
- Linux: 20 Iptables Examples For New SysAdmins
- My 10 UNIX Command Line Mistakes
- 25 PHP Security Best Practices For Sys Admins
- The Novice Guide To Buying A Linux Laptop
- Top 5 Email Client For Linux, Mac OS X, and Windows Users
- Top 20 OpenSSH Server Best Security Practices
- Top 10 Open Source Web-Based Project Management Software
Facebook it - Tweet it - Print it -
We're here to help you make the most of sysadmin work. So, subscribe!
{ 2 comments… read them below or add one }
i am trying to setup multipath on centos 5 and using SAN HP storage disks
problem is /dev/dm-5, /dev/dm-6 and /dev/dm-7 automatically coming after reboot.
i have added three multipath in multipath.conf. i am not able to mount these device /dev/dm-2, /dev/dm-3 and /dev/dm4
why these new device maps automatically added??
[root@BL480 ~]# multipath -l
extravm (360014380024d224a0001000000420000) dm-4 HP,HSV300
[size=60G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:3 sdc 8:32 [active][undef]
\_ 0:0:1:3 sdf 8:80 [active][undef]
joomla (360014380024d224a00010000003c0000) dm-3 HP,HSV300
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:2 sdb 8:16 [active][undef]
\_ 0:0:1:2 sde 8:64 [active][undef]
sqlvm (360014380024d224a0001000000140000) dm-2 HP,HSV300
[size=80G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:1 sda 8:0 [active][undef]
\_ 0:0:1:1 sdd 8:48 [active][undef]
[root@BL480 ~]# fdisk -l
Disk /dev/cciss/c0d0: 36.3 GB, 36380413440 bytes
64 heads, 32 sectors/track, 34695 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/cciss/c0d0p1 * 1 100 102384 83 Linux
/dev/cciss/c0d0p2 101 34695 35425280 8e Linux LVM
Disk /dev/sda: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 1 10443 83883366 83 Linux
Disk /dev/sdb: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 13055 104857568+ 83 Linux
Disk /dev/sdc: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 61440 62914544 83 Linux
Disk /dev/sdd: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 10443 83883366 83 Linux
Disk /dev/sde: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 13055 104857568+ 83 Linux
Disk /dev/sdf: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 61440 62914544 83 Linux
Disk /dev/dm-2: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/dm-2p1 1 10443 83883366 83 Linux
Disk /dev/dm-3: 107.3 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/dm-3p1 1 13055 104857568+ 83 Linux
Disk /dev/dm-4: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/dm-4p1 1 61440 62914544 83 Linux
Disk /dev/dm-5: 85.8 GB, 85896566784 bytes
255 heads, 63 sectors/track, 10442 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/dm-5 doesn’t contain a valid partition table
Disk /dev/dm-6: 107.3 GB, 107374150144 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/dm-6 doesn’t contain a valid partition table
Disk /dev/dm-7: 64.4 GB, 64424493056 bytes
255 heads, 63 sectors/track, 7832 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/dm-7 doesn’t contain a valid partition table
Thanks for this article. But there is a slightly mistake in line “# /etc/init.dmultipathd start” this should be amend as “# /etc/init.d/multipathd start”.
Regards
Charanjit Singh Cheema