Menu

Lumen help

Connecting an Ubuntu bare metal server to Lumen Network Storage

After you have configured a node on a Lumen® Network Storage service, you can connect it to a bare metal server. Below are the steps and terminal commands necessary to mount the node to the server.  


Using a CentOS bare metal server? Learn how to connect a CentOS bare metal server to Network Storage

There are six steps to connect an Ubuntu bare metal server to Network Storage:

Step 1: Update the system and software

  1. Check for updates.
                apt-get update 

apt-get upgrade -y --auto-remove 

# apt-get dist-upgrade -y 

# apt-get autoremove -y 

# apt-get autoclean -y
  1. Install network tools.
                apt install inetutils-traceroute -y 

apt install traceroute -y
  1. Install NFS tools.
                apt install nfs-common -y
  1. Install iSCSI tools.
                apt install multipath-tools -y 

apt install open-iscsi -y 

apt install libiscsi-bin -y 

apt install tgt –y

Step 2: Set up /etc/hosts for Network Storage

Example /etc/hosts for NFS only:

                10.10.10.11      SNC01-NFS1 
10.10.10.12      SNC02-NFS2

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS

Example /etc/hosts for NFS and iSCSI:

                10.10.10.11      SNC01-NFS1 
10.10.10.12      SNC02-NFS2 
10.10.10.16      SNC01-ISCSI1 
10.10.10.15      SNC02-ISCSI1 
10.10.10.14      SNC01-ISCSI2 
10.10.10.13      SNC02-ISCSI2 

#YOUR ASSIGNED IP ADDRESS
10.10.10.21     LOCALHOST-NFS

Step 3: Update the /etc/netplan configuration

Example: 

                cd /etc/netplan/ 
ls -al 
total 12 
drwxr-xr-x   2 root root 4096 Mar  1 17:56 . 
drwxr-xr-x 100 root root 4096 Mar  2 20:52 .. 
-rw-r--r--   1 root root  611 Mar  1 16:51 lumn.yaml 
     vlan.4: 
         id: 4 
         link: bond0 
         mtu: 8900 
         addresses: [10.10.10.21/24] 

Example of entire lumn.yaml: 
network: 
ethernets: 
     switchports: 
         match: 
             name: en* 
bonds: 
     bond0: 
         interfaces: [switchports] 
         mtu: 8900 
         parameters: 
             down-delay: 0 
             lacp-rate: fast 
             mii-monitor-interval: 0 
             mode: 802.3ad 
             transmit-hash-policy: layer2+3 
             up-delay: 0 
vlans: 
     vlan.3: 
         id: 3 
         link: bond0 
         mtu: 8900 
         addresses: [198.91.0.140/29] 
         gateway4: 198.91.0.137 
         nameservers: 
             addresses: 
             - 205.171.3.65 
             - 205.171.2.65 
     vlan.4: 
         id: 4 
         link: bond0 
         mtu: 8900 
         addresses: [10.10.10.21/24] 
#### 
#Try the netplan to verify functionality of network

netplan try
 
#Apply the netplan to make this permanent. 
netplan apply

Step 4: Verify network connectivity by pinging local, NFS, and iSCSI IP addresses

Example: 

                ping -c 3 LOCALHOST-LNS 

PING LOCALHOST-LNS (10.10.10.21) 56(84) bytes of data. 

64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=1 ttl=64 time=0.018 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=2 ttl=64 time=0.022 ms 
64 bytes from LOCALHOST-LNS (10.10.10.21): icmp_seq=3 ttl=64 time=0.023 ms 

--- LOCALHOST-LNS ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2053ms 
rtt min/avg/max/mdev = 0.018/0.021/0.023/0.002 ms 

ping -c 3 SNC01-NFS1 

PING SNC01-NFS1 (10.10.10.11) 56(84) bytes of data. 

64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=1 ttl=64 time=10.7 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=2 ttl=64 time=0.153 ms 
64 bytes from SNC01-NFS1 (10.10.10.11): icmp_seq=3 ttl=64 time=0.093 ms 

--- SNC01-NFS1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2019ms 
rtt min/avg/max/mdev = 0.093/3.638/10.668/4.971 ms 

ping -c 3 SNC02-NFS2 

PING SNC02-NFS2 (10.10.10.12) 56(84) bytes of data. 

64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=1 ttl=64 time=10.6 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=2 ttl=64 time=0.124 ms 
64 bytes from SNC02-NFS2 (10.10.10.12): icmp_seq=3 ttl=64 time=0.099 ms 

--- SNC02-NFS2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2017ms 

rtt min/avg/max/mdev = 0.099/3.605/10.592/4.940 ms 

### Ping the iSCSI IPs 

ping -c 3 SNC01-ISCSI1 

PING SNC01-ISCSI1 (10.10.10.16) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=1 ttl=64 time=16.2 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=2 ttl=64 time=0.122 ms 
64 bytes from SNC01-ISCSI1 (10.10.10.16): icmp_seq=3 ttl=64 time=0.098 ms 

--- SNC01-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2011ms 
rtt min/avg/max/mdev = 0.098/5.474/16.203/7.586 ms 

ping -c 3 SNC02-ISCSI1 

PING SNC02-ISCSI1 (10.10.10.15) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=1 ttl=64 time=44.6 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=2 ttl=64 time=0.143 ms 
64 bytes from SNC02-ISCSI1 (10.10.10.15): icmp_seq=3 ttl=64 time=0.091 ms 

--- SNC02-ISCSI1 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2031ms 
rtt min/avg/max/mdev = 0.091/14.956/44.635/20.985 ms 

ping -c 3 SNC01-ISCSI2 

PING SNC01-ISCSI2 (10.10.10.14) 56(84) bytes of data. 

64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=1 ttl=64 time=10.8 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=2 ttl=64 time=0.136 ms 
64 bytes from SNC01-ISCSI2 (10.10.10.14): icmp_seq=3 ttl=64 time=0.088 ms 

--- SNC01-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2010ms 
rtt min/avg/max/mdev = 0.088/3.663/10.766/5.022 ms 

ping -c 3 SNC02-ISCSI2 

PING SNC02-ISCSI2 (10.10.10.13) 56(84) bytes of data. 

64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=1 ttl=64 time=10.7 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=2 ttl=64 time=0.092 ms 
64 bytes from SNC02-ISCSI2 (10.10.10.13): icmp_seq=3 ttl=64 time=0.092 ms 
  
--- SNC02-ISCSI2 ping statistics --- 

3 packets transmitted, 3 received, 0% packet loss, time 2005ms 
rtt min/avg/max/mdev = 0.092/3.615/10.662/4.982 ms

Step 5: Set up NFS

  1. Show the available mounts. 
                showmount --exports <ip_address/Host_name of the nfs-server>

Example: 

                showmount --exports <ip_address/Host_name of the nfs-server> 

Example output:  

showmount --exports SNC01-NFS1 
Export list for SNC01-NFS1: 
/LNSML4ECTST00009VOL01 (everyone) 
/LNSML4ECTST00009VOL02 (everyone) 
/                      (everyone) 

showmount --exports SNC02-NFS2 

Export list for SNC02-NFS2: 

/LNSML4ECTST00009VOL01 (everyone) 
/LNSML4ECTST00009VOL02 (everyone) 
/                      (everyone)
  1. Create mount points for each volume.
                mkdir -p /nfs1 

mkdir -p /nfs2
  1. Mount a NFS volume. 
                mount -t nfs -o options <nfs-server:/remote/share_directory>  	</local/mount_directory>

Example:

                mount -t nfs -o clientaddr=10.10.10.21 SNC01-NFS1:/LNSML4ECTST00009VOL01 /nfs1 

mount -t nfs -o clientaddr=10.10.10.21 SNC02-NFS2:/LNSML4ECTST00009VOL02 /nfs2
  1. Check for available space. 
                df –BG

Example:

                f -BG 

Filesystem                                     1G-blocks  Used Available Use% Mounted on 

<<SNIP>> 

SNC01-NFS1:/LNSML4ECTST00009VOL01       48G    1G       48G   1% /nfs1 

SNC02-NFS2:/LNSML4ECTST00009VOL02     4750G    1G     4750G   1% /nfs2
  1. Ensure the NFS mounts are writable. 
                cd /nfs1 

dd if=/dev/urandom of=./zz1 bs=10k count=10000 

ls -al 

cd /nfs2 

dd if=/dev/urandom of=./zz2 bs=100k count=10000 

ls –al
  1. If you choose, you can unmount the NFS mount.
                cd /root 

umount /nfs1 

umount /nfs2
  1. Add NFS mount(s) to /etc/fstab to test functionality.

Example:

                SNC01-NFS1:/LNSML4ECTST00009VOL01 /nfs1 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0 

SNC02-NFS2:/LNSML4ECTST00009VOL02 /nfs2 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0
  1. Check FSTAB NFS entries using the mountpoint.

Example:

                mount /nfs1 

mount /nfs2 

df -BG 

Filesystem                                     1G-blocks  Used Available Use% Mounted on 

SNC01-NFS1:/LNSML4ECTST00009VOL01       48G    1G       48G   1% /nfs1 

SNC02-NFS2:/LNSML4ECTST00009VOL02     4750G    2G     4749G   1% /nfs2

Step 6: Set up iSCSI

  1. Show your wwids.
                more /etc/multipath/wwids

Important: wwids will be empty immediately after setup.

  1. Check or fix multiplat.cond.
                more /etc/multipath/wwids

Example:

                defaults { 
    user_friendly_names yes 
}
  1. Set up iSCSI service.
                # Setup 
mkdir /iscsi0 
cd /iscsi0 
ls -al 
df -k 
cd 
systemctl restart iscsid.service
  1. Locate your client machine's IQN located at /etc/iscsi/initiatorname.iscsi. You will need this to add an iSCSI volume in the Network Storage portal (located within Control Center), This value will be used in the Mapped Client IQN field when you create a volume.

Example:

                Use iqn.1993-08.org.debian:01:1928d48ec7e2
  1. Create a volume before proceeding to the next steps.
  1. Discover the target. This will list the volumes you created.
                iscsiadm -m discovery -t sendtargets -p <IP Address/Hostname>

Example:

                SNC01-		ISCSI1 

10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI1 

10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC01-ISCSI2 

10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI2 

10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31
  1. Discover any blocked devices.
                lsblk –e7

Example:

                sblk -e7 

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT 

nvme0n1     259:0    0 447.1G  0 disk 
├─nvme0n1p1 259:1    0   512M  0 part /boot/efi 
└─nvme0n1p2 259:2    0  55.9G  0 part 
  └─os-root 253:0    0  55.9G  0 lvm  / 
nvme1n1     259:3    0 894.3G  0 disk 
nvme2n1     259:4    0 894.3G  0 disk
  1. Use iscsiadmin to list all target information. (When running this command, you may experience timeouts. If this occurs, run the command again.)
                iscsiadm --mode node -l all
  1. Restart iscsi.
                /etc/init.d/open-iscsi restart

Example:

                /etc/init.d/open-iscsi restart 

Restarting open-iscsi (via systemctl): open-iscsi.service.
  1. Run discovery using iscsiadm.

Example:

                iscsiadm -m discovery -t sendtargets -p  

ML4ECTSTS0201SNC01-	ISCSI1 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI1 

10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC01-ISCSI2 

10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t sendtargets -p ML4ECTSTS0201SNC02-ISCSI2 

10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31
  1. Log in to each target name using the above returned targets from iscsiadm.

Example:

                iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.13:3260,1036 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.15:3260,1039 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.16:3260,1037 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.14:3260,1035 --login 
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] (multiple) 
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] successful.
  1. If the following message appears after logging in to each target name, log out of each portal and log in again.
                iscsiadm --mode node --targetname iqn.1992-	08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.13:3260,1036 --login 

iscsiadm: default: 1 session requested, but 1 already present. 

iscsiadm: Could not log into all portals

Example:

                iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.13:3260,1036 --logout 
Logging out of session [sid: 9, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] 
Logout of [sid: 9, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.15:3260,1039 --logout 
Logging out of session [sid: 12, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] 
Logout of [sid: 12, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.16:3260,1037 --logout 
Logging out of session [sid: 11, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] 
Logout of [sid: 11, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] successful. 
iscsiadm --mode node --targetname iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 --portal 10.10.10.14:3260,1035 --logout 
Logging out of session [sid: 10, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] 
Logout of [sid: 10, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260] successful.
  1. Show all sessions.
                iscsiadm --mode session

Example:

                iscsiadm --mode session 

tcp: [17] 10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash) 
tcp: [18] 10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash) 
tcp: [19] 10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash) 
tcp: [20] 10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 (non-flash)
  1. Display information about the nodes.
                iscsiadm -m node -o show | more
  1. Fix /etc/multipath.conf.
                vi /etc/multipath.conf

Example:

                defaults { 
    user_friendly_names yes 
} 
blacklist { 
    devnode "^(ram|zram|raw|loop|fd|md|dm-|sr|scd|dcssblk)[0-9]" 
    devnode "^(td|hd|vd)[a-z]" 
    devnode "^cciss!c[0-9]d[0-9]*" 
}
  1. Rescan iscsiadm.
                iscsiadm -m session --rescan

Example:

                iscsiadm -m session --rescan 

Rescanning session [sid: 17, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.13,3260] 
Rescanning session [sid: 18, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.15,3260] 
Rescanning session [sid: 19, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.16,3260] 
Rescanning session [sid: 20, target: iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31, portal: 10.10.10.14,3260]
  1. Run iscsiadm discovery for all ST devices for each portal.
                iscsiadm -m discovery -t st -p <portal> -o new

Example:

                iscsiadm -m discovery -t st -p 10.10.10.13,3260 -o new 

10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
iscsiadm -m discovery -t st -p 10.10.10.15,3260 -o new 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t st -p 10.10.10.16,3260 -o new 

10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 

iscsiadm -m discovery -t st -p 10.10.10.14,3260 -o new 

10.10.10.14:3260,1035 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.15:3260,1039 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.16:3260,1037 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 
10.10.10.13:3260,1036 iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31
  1. Run iscsi-ls on each portal.
                iscsi-ls -s iscsi://<portal>

Example:

                iscsi-ls -s iscsi://10.10.10.13,3260 

Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
root@ML4ECTSTSB030206:~# iscsi-ls -s iscsi://10.10.10.15,3260 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
root@ML4ECTSTSB030206:~# iscsi-ls -s iscsi://10.10.10.16,3260 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
root@ML4ECTSTSB030206:~# iscsi-ls -s iscsi://10.10.10.14,3260 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.13:3260,1036 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.16:3260,1037 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.15:3260,1039 
Target:iqn.1992-08.com.netapp:sn.b0112cea9be311ec857ed039ea26ae85:vs.31 Portal:10.10.10.14:3260,1035
  1. Verify the partitions.
                cat /proc/partitions
  1. Check the multipaths.
                lsblk -e7 

NAME                                MAJ:MIN RM   SIZE RO TYPE  	MOUNTPOINT 

sda                                   8:0    0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
sdb                                   8:16   0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
sdc                                   8:32   0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
sdd                                   8:48   0   250G  0 disk 
└─3600a098038314630452b532f46714a4f 253:1    0   250G  0 mpath 
nvme0n1                             259:0    0 447.1G  0 disk 
├─nvme0n1p1                         259:1    0   512M  0 part  	/boot/efi 
└─nvme0n1p2                         259:2    0  55.9G  0 part 
  └─os-root                         253:0    0  55.9G  0 lvm   / 
nvme1n1                             259:3    0 894.3G  0 disk 
nvme2n1                             259:4    0 894.3G  0 disk
  1. Check wwids.
                cat /proc/partitions

Example:

                more /etc/multipath/wwids 

# Multipath wwids, Version : 1.0 
# NOTE: This file is automatically maintained by multipath and multipathd. 
# You should not need to edit this file in normal circumstances. 
# 
# Valid WWIDs: 
/3600a098038314630452b532f46714a4c/   ### OLD WWID FOR DM-0 - IGNORE ### 
/3600a098038314630452b532f46714a4f/
  1. Multipath the wwid.
                multipath -f /dev/mapper/<each lun>

Example:

                multipath -f /dev/mapper/3600a098038314630452b532f46714a4f
  1. Reset and list.
                multipath -r 

multipath -ll 

3600a098038314630452b532f46714a4f dm-1 NETAPP,LUN C-Mode 
size=250G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw 
|-+- policy='service-time 0' prio=50 status=active 
| |- 10:0:0:0 sda     8:0   active ready running 
| `- 13:0:0:0 sdd     8:48  active ready running 
`-+- policy='service-time 0' prio=10 status=enabled 
  |- 11:0:0:0 sdb     8:16  active ready running 
  `- 12:0:0:0 sdc     8:32  active ready running
  1. List the devices.
                ls -al /dev/dm-* 

brw-rw---- 1 root disk 253, 0 Mar  4 22:43 /dev/dm-0 

brw-rw---- 1 root disk 253, 1 Mar  7 21:05 /dev/dm-1
  1. Put an ext4 filesystem on dm-1.
                mkfs -t ext4 /dev/dm-#

Example:

                mkfs -t ext4 /dev/dm-1 

mke2fs 1.45.5 (07-Jan-2020) 

Creating filesystem with 65536000 4k blocks and 16384000 inodes 
Filesystem UUID: 2b53636b-d450-4ba3-84ce-c74f4f9ce770 
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872 
Allocating group tables: done 
Writing inode tables: done 
Creating journal (262144 blocks): done 
Writing superblocks and filesystem accounting information: done
  1. Create a mountpoint.
                mkdir /iscsi1
  1. Add the mountpoint to fstab.
                more /etc/fstab 

/dev/dm-1 /iscsi1 ext4 defaults 0 0 

Example including both NFS mountpoints: 

SNC01-NFS1:/LNSML4ECTST00009VOL01 /nfs1 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0 
SNC02-NFS2:/LNSML4ECTST00009VOL02 /nfs2 nfs 			rw,hard,rsize=8192,wsize=8192,timeo=14 0 0 
# iSCSI for on-tap 
/dev/dm-1 /iscsi1 ext4 defaults 0 0
  1. Mount the iSCSI volume.
                mount /iscsi1

Step 7: Display the available storage space

                df -BG 

Filesystem                                     1G-blocks  Used Available Use% Mounted on 

<<SNIP>> 

SNC01-NFS1:/LNSML4ECTST00009VOL01       48G    1G       48G   1% /nfs1 
SNC02-NFS2:/LNSML4ECTST00009VOL02     4750G    2G     4749G   1% /nfs2 
/dev/mapper/3600a098038314630452b532f46714a4f       246G    1G      233G   1% /iscsi1

Step 8: Write test data to /iscsi1

                cd /iscsi1/ 

dd if=/dev/urandom of=./zz1 bs=1000k count=4000 

4000+0 records in 

4000+0 records out 
4096000000 bytes (4.1 GB, 3.8 GiB) copied, 28.1742 s, 145 MB/s 
# Note - /dev/urandom max is about 145 MB/s 
# After some other writes 
df -BG /iscsi1 
Filesystem                                    1G-blocks  Used Available Use% Mounted on 
/dev/mapper/3600a098038314630452b532f46714a4f      246G   43G      191G  19% /iscsi1