Linux Multipathing example

How to configure Multipath IO in CentOS

SAN Storage Configuration

Perform preliminary configuration steps within the storage controller, FC switches e.g.:

LUN Creation
FC zoning
LUN Masking

Install Multipath Driver

yum install device-mapper-multipath
systemctl enable multipathd
systemctl start multipathd

Scan the FC HBA device for new LUNs

The device numbers will likely be different to this example
Make sure you scan both paths (i.e. both HBAs)

[root@servera ~] echo "- - -" > /sys/class/scsi_host11/scan
[root@servera ~] echo "- - -" > /sys/class/scsi_host12/scan

Configure multipathd & LVM

edit /etc/multipath.conf:

These are some of the options I like to use. There are plenty more you can consider if doing bespoke multipath configs.
The blacklist ensures multipathd does not try to configure local disks etc.

defaults {
        user_friendly_names yes
        find_multipaths yes
}
 
blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^(hd|xvd|vd)[a-z]*"
        wwid "*"
}
blacklist_exceptions {
        wwid "35000d3200073ee0000000000000058d2e"
}

edit LVM Config (/etc/lvm/lvm.conf)

This whitelist will prevent LVM from scanning the individual paths
(This assumes you only have a local sda and sdb - otherwise edit accordingly)

filter = [ "a|/dev/sda|", "a|/dev/sdb|", "a|/dev/mapper/mpath.*|", "r|.*|" ]

Checking the device paths

You should now see something such as the below (4 x raw device paths and one mpatha device)

fdisk -l

[root@opsynxsr0401 ~]# fdisk -l
 
Disk /dev/sdb: 599.2 GB, 5998190264 bytes, 11715215 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
 
 
Disk /dev/sda: 199.6 GB, 199446543808 bytes, 387544984 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x000ea788
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2050047     1024000   83  Linux
/dev/sda2         2050048   104452095    51201024   8e  Linux LVM
 
Disk /dev/sdd: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/sdc: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/sdf: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/sde: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/mapper/mpatha: 13194.1 GB, 13194139533312 bytes, 25769803776 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes
 
 
Disk /dev/mapper/vgos-root.fs: 21.0 GB, 20971520000 bytes, 40960000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 
 
Disk /dev/mapper/vgos-var.fs: 8388 MB, 8388608000 bytes, 16384000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 
 
Disk /dev/mapper/vgos-tmp.fs: 5242 MB, 5242880000 bytes, 10240000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
 
 
Disk /dev/mapper/vgdata-data1.fs: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 2097152 bytes / 2097152 bytes

multipath -ll

[root@servera ~]# multipath -ll
mpatha (35000d3200073ee0000000000000058d2f) dm-2 COMPELNT,Compellent Vol
size=12T features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  |- 1:0:1:1  sde 8:64 active ready running
  |- 1:0:3:1  sdf 8:80 active ready running
  |- 12:0:1:1 sdd 8:48 active ready running
  `- 12:0:0:1 sdc 8:32 active ready running

PV Listing

[root@opsynxsr0401 ~]# pvs
  PV                 VG       Fmt  Attr PSize  PFree
  /dev/mapper/mpatha vgdata2   lvm2 a--  12.00t     0
  /dev/sda2          vgos     lvm2 a--  32.23g     0
  /dev/sdb           vgdata   lvm2 a--   2.16t     0

Tag

Back to all post