Lumen help

Connecting a CentOS bare metal server to Lumen Network Storage

  1. Check for updates.
                yum check-update 

yum update -y 
            
                yum -y install traceroute
            
  1. Install NFS tools.
                yum -y install nfs-utils
            
  1. Install iSCSI tools.
  1. Install traceroute.
                rpm -qa |grep device-mapper 

lsmod |grep dm_multipath 

yum -y install device-mapper-multipath 

/sbin/mpathconf --enable 

multipath -ll 

vi /etc/multipath.conf 

chkconfig multipathd on 

yum -y install iscsi-initiator-utils 
            

Example /etc/hosts for NFS only:

                10.10.10.11      SNC01-NFS1
10.10.10.12      SNC02-NFS2

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS
            

Example /etc/hosts for NFS and iSCSI:

                10.10.10.11      SNC01-NFS1
10.10.10.12      SNC02-NFS2
10.10.10.16      SNC01-ISCSI1 
10.10.10.15      SNC02-ISCSI1 
10.10.10.14      SNC01-ISCSI2 
10.10.10.13      SNC02-ISCSI2 

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS
            

Example of bond for vlan4 for IP 10.10.10.21:

                cd /etc/sysconfig/network-scripts 

more ifcfg-bond0.4 

VLAN=yes 
TYPE=Vlan 
PHYSDEV=bond0 
DEVICE=bond0.4 
VLAN_ID=4 
REORDER_HDR=yes 
GVRP=no 
MVRP=no 
HWADDR= 
PROXY_METHOD=none 
BROWSER_ONLY=no 
BOOTPROTO=none 
IPADDR=10.10.10.21 
PREFIX=24 
DEFROUTE=no 
IPV4_FAILURE_FATAL=no 
IPV6INIT=no 
IPV6_AUTOCONF=no 
IPV6_DEFROUTE=no 
IPV6_FAILURE_FATAL=no 
IPV6_ADDR_GEN_MODE=stable-privacy 
NAME=bond0.4 
ONBOOT=no
            
                # Start vlan 4
ifup bond0.4

# If you need stop vlan 4 at a later date
ifdown bond0.4
            

Example:

                ping -c 3 LOCALHOST-LNS 

PING LOCALHOST-LNS (10.10.10.21) 56(84) bytes of data. 

64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=1 ttl=64 time=0.019 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=2 ttl=64 time=0.008 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=3 ttl=64 time=0.006 ms 

--- LOCALHOST-LNS ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.006/0.011/0.019/0.005 ms 

ping -c 3 SNC01-NFS1 

PING SNC01-NFS1 (10.10.10.11) 56(84) bytes of data. 

64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=1 ttl=64 time=0.069 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=2 ttl=64 time=0.064 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=3 ttl=64 time=0.060 ms 

--- SNC01-NFS1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 1999ms 
rtt min/avg/max/mdev = 0.060/0.064/0.069/0.007 ms 

ping -c 3 SNC02-NFS2 

PING SNC02-NFS2 (10.10.10.12) 56(84) bytes of data. 

64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=1 ttl=64 time=0.084 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=2 ttl=64 time=0.072 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=3 ttl=64 time=0.062 ms 

--- SNC02-NFS2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2003ms 
rtt min/avg/max/mdev = 0.062/0.072/0.084/0.013 ms 

ping -c 3 SNC01-ISCSI1 

PING SNC01-ISCSI1 (10.10.10.16) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=1 ttl=64 time=0.080 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=2 ttl=64 time=0.075 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=3 ttl=64 time=0.065 ms 

--- SNC01-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.065/0.073/0.080/0.009 ms 

ping -c 3 SNC02-ISCSI1 

PING SNC02-ISCSI1 (10.10.10.15) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=1 ttl=64 time=0.083 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=2 ttl=64 time=0.075 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=3 ttl=64 time=0.068 ms 

--- SNC02-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.068/0.075/0.083/0.009 ms 

ping -c 3 SNC01-ISCSI2 

PING SNC01-ISCSI2 (10.10.10.14) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=1 ttl=64 time=0.080 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=2 ttl=64 time=0.077 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=3 ttl=64 time=0.058 ms 

--- SNC01-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2000ms 
rtt min/avg/max/mdev = 0.058/0.071/0.080/0.013 ms 

ping -c 3 SNC02-ISCSI2 

PING SNC02-ISCSI2 (10.10.10.13) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=1 ttl=64 time=0.078 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=2 ttl=64 time=0.073 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=3 ttl=64 time=0.065 ms 

--- SNC02-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2002ms 
rtt min/avg/max/mdev = 0.065/0.072/0.078/0.005 ms
            
  1. Show the available mounts.
                showmount --exports <ip_address/Host_name of the nfs-server> 
            

Example: 

                showmount --exports <ip_address/Host_name of the nfs-server> 

Example output:  

showmount --exports SNC01-NFS1 

Export list for SNC01-NFS1: 

/LNS2ML4ECTST00006VOL01 (everyone) 

/LNS2ML4ECTST00006VOL02 (everyone) 

/LNS2ML4ECTST00006VOL03 (everyone) 
            
  1. Create mount points for each volume.
                mkdir -p /nfs1 

mkdir -p /nfs2 

mkdir -p /nfs3 
            
  1. Mount a NFS volume. 
                mount -t nfs -o options <nfs-server:/remote/share_directory>  </local/mount_directory>
            

Example:

                mount -t nfs -o clientaddr=10.10.10.21 SNC01-NFS1:/LNS2ML4ECTST00006VOL01 /nfs1 

mount -t nfs -o clientaddr=10.10.10.21 SNC02-NFS2:/LNS2ML4ECTST00006VOL02 /nfs2 

mount -t nfs -o clientaddr=10.10.10.21 SNC01-NFS1:/LNS2ML4ECTST00006VOL03 /nfs3 

df -BG 

Filesystem                                      1G-blocks  Used Available Use% Mounted on 

SNC01-NFS1:/LNS2ML4ECTST00006VOL01       72G    1G       72G   1% /nfs1 

SNC02-NFS2:/LNS2ML4ECTST00006VOL02       74G    1G       74G   1% /nfs2 

SNC01-NFS1:/LNS2ML4ECTST00006VOL03       87G    1G       87G   1% /nfs3 
            
  1. Ensure the NFS mounts are writable.
                cd /nfs1 

dd if=/dev/urandom of=./zz1 bs=10k count=10000 

ls -al 

cd /nfs2 

dd if=/dev/urandom of=./zz2 bs=100k count=10000 

ls -al 

cd /nfs3 

dd if=/dev/urandom of=./zz3 bs=200k count=10000 

ls -al 
            
  1. If you choose, you can unmount the NFS mount.
                cd /root 

umount /nfs1 

umount /nfs2 

umount /nfs3
            

Example of current mounts in the FSTAB NFS entries:

                Filesystem                                      1G-blocks  Used Available Use% Mounted on 

<SNIP> 

ML4ECTSTS0201SNC01-NFS1:/LNS2ML4ECTST00006VOL01       72G    1G       72G   1% /nfs1 

ML4ECTSTS0201SNC02-NFS2:/LNS2ML4ECTST00006VOL02       74G    1G       73G   2% /nfs2 

ML4ECTSTS0201SNC01-NFS1:/LNS2ML4ECTST00006VOL03       87G    2G       85G   3% /nfs3 
            
  1. Show your wwids.
                more /etc/multipath/wwids
            
  1. Check or fix multiplat.cond.

Example:

                defaults { 

        user_friendly_names yes 

        find_multipaths yes 

} 
            
  1. Set up iSCSI service.
                # Setup 

mkdir /iscsi0 

cd /iscsi0 

ls -al 

df -k 

cd
systemctl restart iscsid.service 
            

Example:

                InitiatorName=iqn.1994-05.com.redhat:fff7ba98df65
            
  1. Discover the target. This will list the volumes you created.
                iscsiadm -m discovery -t sendtargets -p <IP Address/Hostname>
            

Example:

                iscsiadm -m discovery -t sendtargets -p SNC01-ISCSI1 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
iscsiadm -m discovery -t sendtargets -p SNC02-ISCSI1 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
iscsiadm -m discovery -t sendtargets -p SNC01-ISCSI2 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
iscsiadm -m discovery -t sendtargets -p SNC02-ISCSI2 
10.10.10.13:3260,1044 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.14:3260,1043 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.15:3260,1042 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
10.10.10.16:3260,1041 iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39 
            
  1. Discover any blocked devices.
                lsblk -e7
            

Example:

                lsblk -e7 

NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT 

sda       8:0    0   1.8T  0 disk 

sdb       8:16   0   1.8T  0 disk 

sdc       8:32   0   1.8T  0 disk 

sdd       8:48   0   1.8T  0 disk 

sde       8:64   0 223.5G  0 disk 

├─sde1    8:65   0   512M  0 part /boot/efi 

└─sde2    8:66   0   223G  0 part / 

nvme0n1 259:0    0   1.5T  0 disk 

nvme1n1 259:1    0   1.5T  0 disk 
            
  1. Use iscsiadmin to list all target information. (When running this command, you may experience timeouts. If this occurs, run the command again.)

Example:

                iscsiadm --mode node -l all

Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.14,3260] (multiple) 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.16,3260] (multiple) 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.15,3260] (multiple) 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.13,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.14,3260] successful. 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.16,3260] successful. 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.15,3260] successful. 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.711f9e09a90c11ec857ed039ea26ae85:vs.39, portal: 10.10.10.13,3260] successful. 
            

Example:

                lsblk -e7 

NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT 

sda       8:0    0   1.8T  0 disk 

sdb       8:16   0   1.8T  0 disk 

sdc       8:32   0   1.8T  0 disk 

sdd       8:48   0   1.8T  0 disk 

sde       8:64   0 223.5G  0 disk 

├─sde1    8:65   0   512M  0 part /boot/efi 

└─sde2    8:66   0   223G  0 part / 

sdf       8:80   0   105G  0 disk 

sdg       8:96   0   105G  0 disk 

sdh       8:112  0   105G  0 disk 

sdi       8:128  0   105G  0 disk 

nvme0n1 259:0    0   1.5T  0 disk 

nvme1n1 259:1    0   1.5T  0 disk 
            
  1. Register with NetApp.

Example:

                rpm -ivh netapp_linux_unified_host_utilities-7-1.x86_64.rpm 

Preparing...                          ################################# [100%] 

Warning: libnl.so library not found, some sanlun commands may not work. Refer Linux Host Utilities Installation and Set up Guide for more details 
Warning: libHBAAPI.so library not found, some sanlun commands may not work. Refer Linux Host Utilities Installation and Setup Guide for more details 
Updating / installing... 
1:netapp_linux_unified_host_utiliti################################# [100%] 
            
  1. Restart, enable, and start multipath.
                systemctl restart iscsid.service 

systemctl enable multipathd 

systemctl start multipathd
            
  1. Recheck multipath.
                multipath -r 

reload: 3600a098038314630452b532f46714a50 undef NETAPP  ,LUN C-Mode 

size=105G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=undef 

|-+- policy='service-time 0' prio=50 status=undef 

| |- 18:0:0:0 sdg     8:96  active ready running 

| `- 21:0:0:0 sdh     8:112 active ready running 

`-+- policy='service-time 0' prio=10 status=undef 

  |- 19:0:0:0 sdf     8:80  active ready running 

  `- 20:0:0:0 sdi     8:128 active ready running 

multipath -ll 

3600a098038314630452b532f46714a50 dm-0 NETAPP  ,LUN C-Mode 

size=105G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw 

|-+- policy='service-time 0' prio=50 status=enabled 

| |- 18:0:0:0 sdg     8:96  active ready running 

| `- 21:0:0:0 sdh     8:112 active ready running 

`-+- policy='service-time 0' prio=10 status=enabled 

  |- 19:0:0:0 sdf     8:80  active ready running 

  `- 20:0:0:0 sdi     8:128 active ready running 

 

lsblk -e7 

NAME                                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT 

sda                                   8:0    0   1.8T  0 disk 

sdb                                   8:16   0   1.8T  0 disk 

sdc                                   8:32   0   1.8T  0 disk 

sdd                                   8:48   0   1.8T  0 disk 

sde                                   8:64   0 223.5G  0 disk 

├─sde1                                8:65   0   512M  0 part  /boot/efi 

└─sde2                                8:66   0   223G  0 part  / 

sdf                                   8:80   0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

sdg                                   8:96   0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

sdh                                   8:112  0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

sdi                                   8:128  0   105G  0 disk 

└─3600a098038314630452b532f46714a50 253:0    0   105G  0 mpath 

nvme0n1                             259:0    0   1.5T  0 disk 

nvme1n1                             259:1    0   1.5T  0 disk 

more /etc/multipath/wwids
            
  1. Set the multipath.
                multipath -f /dev/mapper/<WWID> 

Important: The multipath wwids are located in /etc/multipath/wwids.  

Example:  

multipath -f /dev/mapper/3600a098038314630452b532f46714a50 

 

multipath -r 

 

Example output: 

create: 3600a098038314630452b532f46714a50 undef NETAPP  ,LUN C-Mode 

size=105G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=undef 

|-+- policy='service-time 0' prio=50 status=undef 

| |- 18:0:0:0 sdg     8:96  undef ready running 

| `- 21:0:0:0 sdh     8:112 undef ready running 

`-+- policy='service-time 0' prio=10 status=undef 

  |- 19:0:0:0 sdf     8:80  undef ready running 

  `- 20:0:0:0 sdi     8:128 undef ready running 

 

Example output: 

3600a098038314630452b532f46714a50 dm-0 NETAPP  ,LUN C-Mode 

size=105G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw 

|-+- policy='service-time 0' prio=50 status=enabled 

| |- 18:0:0:0 sdg     8:96  active ready running 

| `- 21:0:0:0 sdh     8:112 active ready running 

`-+- policy='service-time 0' prio=10 status=enabled 

  |- 19:0:0:0 sdf     8:80  active ready running 

  `- 20:0:0:0 sdi     8:128 active ready running 

 

Example devices for multipath:  

ls -al /dev/dm* 

brw-rw----. 1 root disk 253, 0 Mar 23 21:43 /dev/dm-0 

ls -al /dev/mapper/3600a098038314630452b532f46714a50 

lrwxrwxrwx. 1 root root 7 Mar 23 21:43 /dev/mapper/3600a098038314630452b532f46714a50 -> ../dm-0 
            
  1. Put an ext4 filesystem on /dev.dm-0.
                mkfs -t ext4 /dev/dm-0 
            
  1. Create a mountpoint.
                mkfs -t ext4 /dev/dm-0 
            
  1. Add the mountpoint to fstab.
                more /etc/fstab 
            
  1. Mount the iSCSI volume.
                mount /iscsi0 
            
                df -BG 

Filesystem                                      1G-blocks  Used Available Use% Mounted on 

<SNIP> 

/dev/mapper/3600a098038314630452b532f46714a50        104G    1G       98G   1% /iscsi0 
            
                cd /iscsi0/ 

dd if=/dev/urandom of=./zz0 bs=1000k count=4000 

4000+0 records in 

4000+0 records out 

4096000000 bytes (4.1 GB) copied, 23.7238 s, 173 MB/s 

# Note - /dev/urandom max is about ~150 MB/s 

df -BG /iscsi0 

Filesystem                                    1G-blocks  Used Available Use% Mounted on 

/dev/mapper/3600a098038314630452b532f46714a50      104G    4G       95G   4% /iscsi0