SC40220 - iscsi - 2 * FD - Unable to discover through 2nd portal IP

Hi there

Trying to get multipath and switch fabric resilience working with a dual FD setup on the SC4020. FDs are ISCSI1 and ISCSI2.

SAN IP addresses

ISCSI1 IP address 10.154.167.205/24

Controller X p1 10.154.167.201/24

Controller Y p1 10.154.167.203/24

ISCSI2 IP address 10.154.168.206/24

Controller X p2 10.154.168.202/24

Controller Y p2 10.154.168.204/24

 

As you can see I have the 2 FDs spanning across the SAN controllers, i.e port 1 in ISCSI1 & port 2 in ISCSI2 on both controllers. The rationale behind this is that I should be able to maintain multipath operation even if I lose an entire controller on the SC4020.

Each FD and associated controller ports has been addresses as above, each FD has its own /24 and own VLAN. All switchports are in access mode on a juniper EX4550 stack in the appropriate VLAN and are running jumbo frames.

On the linux client side (oracle UL 7 - 3.8.13-35.3.1.el7uek.x86_64) we have two interfaces, em4 and p3p2 which are also addressed in the correct subnets, em4 is 10.154.167.51/24 and p3p2 is 10.154.168.151/24

The client can successfully ping the two member port IP addresses as well as the FD ip address when restricted to the correct source interface.

e.g. from em4 for ISCSI1

root@pd-tk-db01 ~ # ping -I em4 -c1 10.154.167.201
PING 10.154.167.201 (10.154.167.201) from 10.154.167.51 em4: 56(84) bytes of data.
64 bytes from 10.154.167.201: icmp_seq=1 ttl=64 time=0.063 ms

--- 10.154.167.201 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.063/0.063/0.063/0.000 ms
root@pd-tk-db01 ~ # ping -I em4 -c1 10.154.167.203
PING 10.154.167.203 (10.154.167.203) from 10.154.167.51 em4: 56(84) bytes of data.
64 bytes from 10.154.167.203: icmp_seq=1 ttl=64 time=0.067 ms

--- 10.154.167.203 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.067/0.067/0.067/0.000 ms
root@pd-tk-db01 ~ # ping -I em4 -c1 10.154.167.205
PING 10.154.167.205 (10.154.167.205) from 10.154.167.51 em4: 56(84) bytes of data.
64 bytes from 10.154.167.205: icmp_seq=1 ttl=64 time=0.125 ms

--- 10.154.167.205 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.125/0.125/0.125/0.000 ms
root@pd-tk-db01 ~ #

Ditto for p3p2 for ISCSI2

root@pd-tk-db01 ~ # ping -I p3p2 -c1 10.154.168.202
PING 10.154.168.202 (10.154.168.202) from 10.154.168.151 p3p2: 56(84) bytes of d                 ata.
64 bytes from 10.154.168.202: icmp_seq=1 ttl=64 time=0.123 ms

--- 10.154.168.202 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.123/0.123/0.123/0.000 ms
root@pd-tk-db01 ~ # ping -I p3p2 -c1 10.154.168.204
PING 10.154.168.204 (10.154.168.204) from 10.154.168.151 p3p2: 56(84) bytes of d                 ata.
64 bytes from 10.154.168.204: icmp_seq=1 ttl=64 time=0.113 ms

--- 10.154.168.204 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.113/0.113/0.113/0.000 ms
root@pd-tk-db01 ~ # ping -I p3p2 -c1 10.154.168.206
PING 10.154.168.206 (10.154.168.206) from 10.154.168.151 p3p2: 56(84) bytes of d                 ata.
64 bytes from 10.154.168.206: icmp_seq=1 ttl=64 time=0.116 ms

--- 10.154.168.206 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.116/0.116/0.116/0.000 ms
root@pd-tk-db01~ #

So basic network connectivity is fine.

iscsi configured on client as follows:

iscsiadm -m iface -I ISCSI1 -o new
iscsiadm -m iface -I ISCSI2 -o new
iscsiadm -m iface -I ISCSI1 -o update -n iface.net_ifacename -v em4
iscsiadm -m iface -I ISCSI2 -o update -n iface.net_ifacename -v p2p3

If you're still with me by now, the problem is this. Anything done to ISCSI1 targets is fine. Discovery works as expected.

root@pd-tk-db01 ~ # iscsiadm -m discovery -t st -p 10.154.167.205 -I ISCSI1
10.154.167.205:3260,0 iqn.2002-03.com.compellent:5000d31000ac201c
10.154.167.205:3260,0 iqn.2002-03.com.compellent:5000d31000ac201d
10.154.167.205:3260,0 iqn.2002-03.com.compellent:5000d31000ac201e
10.154.167.205:3260,0 iqn.2002-03.com.compellent:5000d31000ac201f
root@pd-tk-db01 ~ #

but not for ISCSI2

root@pd-tk-db01 ~ # iscsiadm -m discovery -t st -p 10.154.168.206 -I ISCSI2
iscsiadm: No portals found

Remember I can ping 10.154.168.206, *** I can even telnet to it on 3260 to prove TCP connectivity is good, but it won't allow me to discover anything through it.

When I added the server definition on the SAN and attempt to add the two HBAs the one corresponding to em4/ISCSI1 shows as up with the correct initiator; but with 4 virtual controller ports, 2 per physical controller port in ICSI1. This worries me, I'd only expect 1 per physical port as each client HBA can only see two physical ports in total.

I can add a HBA corresponding to p3p2, but it never comes up.

We're not permissioned for the compellent portal yet so can't get at any white papers or reference setups that may exist in there, but surely what we're trying is a pretty basic setup and we shouldn't be struggling with it.