Installation/configuration iscsi and multipath on RHEL7
12 二月, 2019 1 則迴響
Refer to : https://access.redhat.com/solutions/894163
Refer to : https://zhuanlan.zhihu.com/p/26714897
Lab envirement:
OS Verson: RHEL7.4 , kernel 3.10.0-693.21.1.el7.x86_64
LAB1:Install and config iscsi-target and iscsi-initator
LAB2:Install and config Multipath
We are going to start the LAB1:iSCSI
1.we have two server, listed as below:
-
rhel7-iscsi-target 192.168.1.204
-
-
/dev/sdb <— here is our block device
-
-
rhel7-iscsi-initator 192.168.1.206
2.start to install iscsi packages in rhel7-iscsi-target
[root@rhel7-iscci-target ~]# yum install targetcli
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Resolving Dependencies
–> Running transaction check
—> Package targetcli.noarch 0:2.1.fb46-1.el7 will be installed
–> Processing Dependency: python-rtslib >= 2.1.fb41 for package: targetcli-2.1.fb46-1.el7.noarch
–> Processing Dependency: python-configshell for package: targetcli-2.1.fb46-1.el7.noarch
–> Running transaction check
—> Package python-configshell.noarch 1:1.1.fb23-3.el7 will be installed
–> Processing Dependency: pyparsing for package: 1:python-configshell-1.1.fb23-3.el7.noarch
–> Processing Dependency: python-six for package: 1:python-configshell-1.1.fb23-3.el7.noarch
–> Processing Dependency: python-urwid for package: 1:python-configshell-1.1.fb23-3.el7.noarch
—> Package python-rtslib.noarch 0:2.1.fb63-2.el7 will be installed
–> Processing Dependency: python-kmod for package: python-rtslib-2.1.fb63-2.el7.noarch
–> Running transaction check
—> Package pyparsing.noarch 0:1.5.6-9.el7 will be installed
—> Package python-kmod.x86_64 0:0.9-4.el7 will be installed
—> Package python-six.noarch 0:1.9.0-2.el7 will be installed
—> Package python-urwid.x86_64 0:1.1.1-3.el7 will be installed
–> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================
Installing:
targetcli noarch 2.1.fb46-1.el7 rhel-7-server-rpms 64 k
Installing for dependencies:
pyparsing noarch 1.5.6-9.el7 rhel-7-server-rpms 94 k
python-configshell noarch 1:1.1.fb23-3.el7 rhel-7-server-rpms 67 k
python-kmod x86_64 0.9-4.el7 rhel-7-server-rpms 57 k
python-rtslib noarch 2.1.fb63-2.el7 rhel-7-server-rpms 97 k
python-six noarch 1.9.0-2.el7 rhel-7-server-rpms 29 k
python-urwid x86_64 1.1.1-3.el7 rhel-7-server-rpms 654 k
Transaction Summary
============================================================================================================================================
Install 1 Package (+6 Dependent packages)
Total download size: 1.0 M
Installed size: 4.7 M
Is this ok [y/d/N]: y
3.load the target modules
[root@rhel7-iscci-target ~]# systemctl start target
[root@rhel7-iscci-target ~]# systemctl enable target
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.
4.ours block device is /dev/sdb
[root@rhel7-iscci-target ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
5.Enter the targetcli shell.
[root@rhel7-iscci-target ~]# targetcli
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type ‘help’.
/>
6.Start by add your device under /backsstores/block. I’m using the lu0 for my already existing blockdevice(/dev/sda):
/> cd /backstores/block
/backstores/block> create lun0 /dev/sdb
Created block storage object lun0 using /dev/sdb.
/backstores/block>
7.Switch to the /iscsi and create a new iqn. The iqn will be create4d randomly if you do not specify one.
/backstores/block> cd /iscsi
/iscsi> create
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi>
8.Change to your iqn directory and then to the tpg1 directory
/iscsi> ls
o- iscsi ……………………………………………………………………………………………….. [Targets: 1]
o- iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804 ………………………………………… [TPGs: 1]
o- tpg1 ……………………………………………………………………………………. [no-gen-acls, no-auth]
o- acls ……………………………………………………………………………………………… [ACLs: 0]
o- luns ……………………………………………………………………………………………… [LUNs: 0]
o- portals ………………………………………………………………………………………… [Portals: 1]
o- 0.0.0.0:3260 …………………………………………………………………………………………. [OK]
/iscsi> cd iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804/tpg1/
/iscsi/iqn.20…e813c804/tpg1>
9.Obtian you initator name from client (rhel7-iscsi-initator)
9.1 login to your rhel7-iscsi-initator and install iscsi-initiator-utils package
[root@rhel7-iscsi-initator ~]# yum install iscsi-initiator-utils
Dependencies Resolved
============================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================
Installing:
iscsi-initiator-utils x86_64 6.2.0.874-4.el7 rhel-7-server-rpms 420 k
Installing for dependencies:
iscsi-initiator-utils-iscsiuio x86_64 6.2.0.874-4.el7 rhel-7-server-rpms 90 k
Transaction Summary
============================================================================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 510 k
Installed size: 2.5 M
Is this ok [y/d/N]:
9.2 Obtian you initator name from client (rhel7-iscsi-initator)
[root@rhel7-iscsi-initator ~]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:466722ae5ba
[root@rhel7-iscsi-initator ~]#
9.3 start and enable iscsid
[root@rhel7-iscsi-initator ~]# systemctl enable iscsid.service
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
[root@rhel7-iscsi-initator ~]# systemctl start iscsid.service
10.setup acls for the hosts that can login to this portal under /acls, go back to the “rhel7-iscsi-target”
/iscsi/iqn.20…e813c804/tpg1> cd acls
/iscsi/iqn.20…804/tpg1/acls> create iqn.1994-05.com.redhat:466722ae5ba
Created Node ACL for iqn.1994-05.com.redhat:466722ae5ba
/iscsi/iqn.20…804/tpg1/acls>
11.Create a portal. This will be the IP address that we will listen on,go back to the “rhel7-iscsi-target”
/iscsi/iqn.20…804/tpg1/acls> cd ../portals/
/iscsi/iqn.20…/tpg1/portals> create
Using default IP port 3260
Binding to INADDR_ANY (0.0.0.0)
This NetworkPortal already exists in configFS
/iscsi/iqn.20…/tpg1/portals> delete 0.0.0.0 3260
Deleted network portal 0.0.0.0:3260
/iscsi/iqn.20…/tpg1/portals> create
Using default IP port 3260
Binding to INADDR_ANY (0.0.0.0)
Created network portal 0.0.0.0:3260.
/iscsi/iqn.20…/tpg1/portals>
12.Add the lun we reated in step 6 to this target
/iscsi/iqn.20…/tpg1/portals> cd ..
/iscsi/iqn.20…e813c804/tpg1> cd luns
/iscsi/iqn.20…804/tpg1/luns> ls
o- luns …………………………………………………………………………………………………… [LUNs: 0]
/iscsi/iqn.20…804/tpg1/luns> create /backstores/block/lun0
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.1994-05.com.redhat:466722ae5ba
/iscsi/iqn.20…804/tpg1/luns>
13.Check your configuration
/iscsi/iqn.20…804/tpg1/luns> cd ../..
/iscsi/iqn.20….b4ebe813c804> ls
o- iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804 ………………………………………….. [TPGs: 1]
o- tpg1 ……………………………………………………………………………………… [no-gen-acls, no-auth]
o- acls ……………………………………………………………………………………………….. [ACLs: 1]
| o- iqn.1994-05.com.redhat:466722ae5ba …………………………………………………………….. [Mapped LUNs: 1]
| o- mapped_lun0 ………………………………………………………………………….. [lun0 block/lun0 (rw)]
o- luns ……………………………………………………………………………………………….. [LUNs: 1]
| o- lun0 ………………………………………………………………… [block/lun0 (/dev/sdb) (default_tg_pt_gp)]
o- portals ………………………………………………………………………………………….. [Portals: 1]
o- 0.0.0.0:3260 …………………………………………………………………………………………… [OK]
/iscsi/iqn.20….b4ebe813c804>
14.Finally, save your configurations and exit
/iscsi/iqn.20….b4ebe813c804> cd /
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
[root@rhel7-iscci-target ~]#
15.Now, we change to iscsi-initator to run a discobery against the target to verify your iqn is available.
[root@rhel7-iscsi-initator ~]# iscsiadm –mode discoverydb –type sendtargets –portal 192.168.1.204 –discover 192.168.1.204:3261,1 iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804
192.168.1.204:3260,1 iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804
16.Login to this target
[root@rhel7-iscsi-initator ~]# iscsiadm –mode node –targetname iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804 –portal 192.168.1.204 –login
Logging in to [iface: default, target: iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804, portal: 192.168.1.204,3260] (multiple)
Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804, portal: 192.168.1.204,3260] successful.
17.Verify your new device is available
[root@rhel7-iscsi-initator ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: VMware Model: Virtual disk Rev: 2.0
Type: Direct-Access ANSI SCSI revision: 06
Host: scsi33 Channel: 00 Id: 00 Lun: 00
Vendor: LIO-ORG Model: lun0 Rev: 4.0
Type: Direct-Access ANSI SCSI revision: 05
[root@rhel7-iscsi-initator ~]# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00008d56
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 83886079 40893440 8e Linux LVM
Disk /dev/mapper/rhel-root: 39.7 GB, 39720058880 bytes, 77578240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/rhel-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33550336 bytes
19.another job, because we want to demo multipath so that I need create another iscsi path, the IP is 192.168.1.205
[root@rhel7-iscsi-initator ~]# iscsiadm –mode discoverydb –type sendtargets –portal 192.168.1.205 –discover 192.168.1.205:3261,1 iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804
192.168.1.205:3260,1 iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804
[root@rhel7-iscsi-initator ~]# iscsiadm –mode node –targetname iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804 –portal 192.168.1.205 –login
Logging in to [iface: default, target: iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804, portal: 192.168.1.205,3260] (multiple)
Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.rhel7-iscci-target.x8664:sn.b4ebe813c804, portal: 192.168.1.205,3260] successful.
20.Verify your device is available again, you will see two disk in same lun0
[root@rhel7-iscsi-initator ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: VMware Model: Virtual disk Rev: 2.0
Type: Direct-Access ANSI SCSI revision: 06
Host: scsi33 Channel: 00 Id: 00 Lun: 00
Vendor: LIO-ORG Model: lun0 Rev: 4.0
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi34 Channel: 00 Id: 00 Lun: 00
Vendor: LIO-ORG Model: lun0 Rev: 4.0
Type: Direct-Access ANSI SCSI revision: 05
[root@rhel7-iscsi-initator ~]# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00008d56
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 83886079 40893440 8e Linux LVM
Disk /dev/mapper/rhel-root: 39.7 GB, 39720058880 bytes, 77578240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/rhel-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33550336 bytes
Disk /dev/sdc: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33550336 bytes
[root@rhel7-iscsi-initator ~]#
—For now, we already create two path for iscsi-initator. Next we will start to install multipath and config multipath in rhel-iscsi-initator server.—
We are going to start the LAB2:Multipath
refer to : https://access.redhat.com/solutions/66281
1.install Device-mapper-multipath (DM multipath) package
[root@rhel7-iscsi-initator ~]# yum install device-mapper-multipath
============================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================
Installing:
device-mapper-multipath x86_64 0.4.9-111.el7_4.2 rhel-7-server-rpms 134 k
Installing for dependencies:
boost-system x86_64 1.53.0-27.el7 rhel-7-server-rpms 40 k
boost-thread x86_64 1.53.0-27.el7 rhel-7-server-rpms 57 k
device-mapper-multipath-libs x86_64 0.4.9-111.el7_4.2 rhel-7-server-rpms 252 k
librados2 x86_64 1:0.94.5-2.el7 rhel-7-server-rpms 1.7 M
Transaction Summary
============================================================================================================================================
Install 1 Package (+4 Dependent packages)
Total download size: 2.1 M
Installed size: 6.0 M
Is this ok [y/d/N]:
2.Create multipath.conf, we have two way to create multipath.conf
2.1#mpathconf —enable —with_multipathd y
2.2cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/multipath.conf
3.we need find out wwid like as below:
use lsscsi -i
[root@rhel7-iscsi-initator ~]# lsscsi -i
[0:0:0:0] disk VMware Virtual disk 2.0 /dev/sda –
[33:0:0:0] disk LIO-ORG lun0 4.0 /dev/sdb 36001405e01a53e31a2a4705bff8630bc
[34:0:0:0] disk LIO-ORG lun0 4.0 /dev/sdc 36001405e01a53e31a2a4705bff8630bc
fdisk -l
Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33550336 bytes
Disk /dev/sdc: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33550336 bytes
Disk /dev/mapper/mpatha: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33550336 bytes
lsblk
[root@rhel7-iscsi-initator ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 40G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 39G 0 part
├─rhel-root 253:0 0 37G 0 lvm /
└─rhel-swap 253:1 0 2G 0 lvm [SWAP]
sdb 8:16 0 20G 0 disk
└─mpatha 253:2 0 20G 0 mpath
sdc 8:32 0 20G 0 disk
└─mpatha 253:2 0 20G 0 mpath
[root@rhel7-iscsi-initator ~]# cat /sys/block/sda/device/vendor
VMware
[root@rhel7-iscsi-initator ~]# cat /sys/block/sdb/device/vendor
LIO-ORG
[root@rhel7-iscsi-initator ~]#
To view all attached SCSI devices
[root@rhel7-iscsi-initator ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: VMware Model: Virtual disk Rev: 2.0
Type: Direct-Access ANSI SCSI revision: 06
Host: scsi33 Channel: 00 Id: 00 Lun: 00
Vendor: LIO-ORG Model: lun0 Rev: 4.0
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi34 Channel: 00 Id: 00 Lun: 00
Vendor: LIO-ORG Model: lun0 Rev: 4.0
Type: Direct-Access ANSI SCSI revision: 05
To determine vendor and model (product) strings for a specific SCSI disk device (replace sda with the device in question):
[root@rhel7-iscsi-initator ~]# cat /sys/block/sda/device/vendor
VMware
[root@rhel7-iscsi-initator ~]# cat /sys/block/sda/device/model
Virtual disk
[root@rhel7-iscsi-initator ~]# cat /sys/block/sdb/device/vendor
LIO-ORG
[root@rhel7-iscsi-initator ~]# cat /sys/block/sdb/device/model
lun0
4.we can use vim/vi to edit /etc/multipath.conf you want.
defaults {
polling_interval 10
path_selector “round-robin 0″
path_grouping_policy multibus
uid_attribute ID_SERIAL
prio alua
rr_min_io 100
max_fds 8192
rr_weight priorities
failback immediate
no_path_retry fail
user_friendly_names yes
}
blacklist {
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode “^hd[a-z]"
}
multipaths {
multipath {
wwid 36001405e01a53e31a2a4705bff8630bc
alias iscsilab
}
}
5.Check multipath status. to use multipath -ll
root@rhel7-iscsi-initator ~]# multipath -ll
iscsilab (36001405e01a53e31a2a4705bff8630bc) dm-2 LIO-ORG ,lun0
size=20G features=’0′ hwhandler=’0′ wp=rw
`-+- policy=’round-robin 0′ prio=50 status=active
|- 33:0:0:0 sdb 8:16 active ready running
`- 34:0:0:0 sdc 8:32 active ready running
*If you have previously created a multipath device without using the find_multipaths parameter and then you later set the parameter to yes, you may need to remove the WWIDs of any device you do not want created as a multipath device from the /etc/multipath/wwids file. The following shows a sample /etc/multipath/wwids file. The WWIDs are enclosed by slashes (/):
[root@rhel7-iscsi-initator ~]# cat /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/36001405e01a53e31a2a4705bff8630bc/
6.to mount the multipath disk
[root@rhel7-iscsi-initator ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Oct 31 22:44:37 2017
#
# Accessible filesystems, by reference, are maintained under ‘/dev/disk’
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/rhel-root / xfs defaults 0 0
UUID=994ab371-55a5-48bc-a6bc-7d1fbc547021 /boot xfs defaults 0 0
/dev/mapper/rhel-swap swap swap defaults 0 0
/dev/mapper/iscsilab /multipath_lab xfs defaults 0 0
[root@rhel7-iscsi-initator ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rhel-root 37G 1.5G 36G 4% /
devtmpfs 910M 0 910M 0% /dev
tmpfs 920M 0 920M 0% /dev/shm
tmpfs 920M 8.6M 912M 1% /run
tmpfs 920M 0 920M 0% /sys/fs/cgroup
/dev/sda1 1014M 185M 830M 19% /boot
tmpfs 184M 0 184M 0% /run/user/0
/dev/mapper/iscsilab 20G 33M 20G 1% /multipath_lab
7.Check Command
multipathd -k
>show config
>list paths
>list|show status
>show topology
HunterFeng's Blog 