Robbie Cannings

Linux Multipathing example

Robbie Cannings -

How to configure Multipath IO in CentOS

SAN Storage Configuration

Perform preliminary configuration steps within the storage controller, FC switches e.g.:

  • LUN Creation
  • FC zoning
  • LUN Masking

Install Multipath Driver

yum install device-mapper-multipath
systemctl enable multipathd
systemctl start multipathd

Scan the FC HBA device for new LUNs

  • The device numbers will likely be different to this example

  • Make sure you scan both paths (i.e. both HBAs)

    [root@servera ~] echo "- - -" > /sys/class/scsi_host11/scan
    [root@servera ~] echo "- - -" > /sys/class/scsi_host12/scan

Configure multipathd & LVM

edit /etc/multipath.conf:

These are some of the options I like to use. There are plenty more you can consider if doing bespoke multipath configs.
The blacklist ensures multipathd does not try to configure local disks etc.

defaults {
        user_friendly_names yes
        find_multipaths yes
}
 
blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^(hd|xvd|vd)[a-z]*"
        wwid "*"
}
blacklist_exceptions {
        wwid "35000d3200073ee0000000000000058d2e"
}
edit LVM Config (/etc/lvm/lvm.conf)

This whitelist will prevent LVM from scanning the individual paths
(This assumes you only have a local sda and sdb - otherwise edit accordingly)

filter = [ "a|/dev/sda|", "a|/dev/sdb|", "a|/dev/mapper/mpath.*|", "r|.*|" ]

Checking the device paths

You should now see something such as the below (4 x raw device paths and one mpatha device)

fdisk -l
[root@opsynxsr0401 ~]# fdisk -l
 
Disk /dev/sdb: 599.2 GB, 5998190264 bytes, 11715215 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
 
 
Disk /dev/sda: 199.6 GB, 199446543808 bytes, 387544984 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x000ea788
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2050047     1024000   83  Linux
/dev/sda2         2050048   104452095    51201024   8e  Linux LVM
 
Disk /dev/sdd: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/sdc: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/sdf: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/sde: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/mapper/mpatha: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/mapper/vgos-root.fs: 21.0 GB, 20971520000 bytes, 40960000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 
 
Disk /dev/mapper/vgos-var.fs: 8388 MB, 8388608000 bytes, 16384000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 
 
Disk /dev/mapper/vgos-tmp.fs: 5242 MB, 5242880000 bytes, 10240000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 
 
Disk /dev/mapper/vgdata-data1.fs: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
multipath -ll
[root@servera ~]# multipath -ll
mpatha (35000d3200073ee0000000000000058d2f) dm-2 COMPELNT,Compellent Vol
size=12T features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  |- 1:0:1:1  sde 8:64 active ready running
  |- 1:0:3:1  sdf 8:80 active ready running
  |- 12:0:1:1 sdd 8:48 active ready running
  `- 12:0:0:1 sdc 8:32 active ready running
PV Listing
[root@opsynxsr0401 ~]# pvs
  PV                 VG       Fmt  Attr PSize  PFree
  /dev/mapper/mpatha vgdata2   lvm2 a--  12.00t     0
  /dev/sda2          vgos     lvm2 a--  32.23g     0
  /dev/sdb           vgdata   lvm2 a--   2.16t     0