Unsolved
5 Posts
0
208
November 8th, 2022 06:00
Using two FN410s in FX2s to connect blades (ESXi Hosts) to network (Cisco Fiber Hubs)
I am trying to configure two FN410s IOMs in an FX2s chassis to be the main data/network connections for 4 ESXi Hosts that are housed on the blades within the FX2s chassis. This would mean 8 total connections, 4 on 1 switch and 4 on another for redundancy purposes. (Current config is a 1G connection)
Currently, each FN410s IOM is configured in standalone mode. All 4 external SFP+ ports are configured in the default port-channel/LAG (128) for both modules. All 4 connections from 'A1' (Te0/9-12) are connected to Cisco Fiber Hub 1 (Te1/0/1-4), and all 4 connections from 'A2' (Te0/9-12) are connected to Cisco Fiber Hub 2 (Te1/0/1-4). The port-channel/etherchannel is configured on both Cisco Fiber Hubs 1 and 2. Here is what the configurations looks like:
Dell FN410s IOMs External Ports and Port-channel:
!
interface TenGigabitEthernet 0/9
description Connected to "Cisco Fiber Hub 1" Port Te1/0/1
mtu 12000
speed 10000
flowcontrol rx on tx off
!
port-channel-protocol LACP
port-channel 128 mode active
!
protocol lldp
advertise management-tlv management-address system-name
no advertise dcbx-tlv ets-reco
dcbx port-role auto-upstream
no shutdown
!
interface TenGigabitEthernet 0/10
description Connected to "Cisco Fiber Hub 1" Port Te1/0/2
mtu 12000
speed 10000
flowcontrol rx on tx off
!
port-channel-protocol LACP
port-channel 128 mode active
!
protocol lldp
advertise management-tlv management-address system-name
no advertise dcbx-tlv ets-reco
dcbx port-role auto-upstream
no shutdown
!
interface TenGigabitEthernet 0/11
description Connected to "Cisco Fiber Hub 1" Port Te1/0/3
mtu 12000
speed 10000
flowcontrol rx on tx off
!
port-channel-protocol LACP
port-channel 128 mode active
!
protocol lldp
advertise management-tlv management-address system-name
no advertise dcbx-tlv ets-reco
dcbx port-role auto-upstream
no shutdown
!
interface TenGigabitEthernet 0/12
description Connected to "Cisco Fiber Hub 1" Port Te1/0/4
mtu 12000
speed 10000
flowcontrol rx on tx off
!
port-channel-protocol LACP
port-channel 128 mode active
!
protocol lldp
advertise management-tlv management-address system-name
no advertise dcbx-tlv ets-reco
dcbx port-role auto-upstream
no shutdown
!
interface ManagementEthernet 0/0
ip address (redacted)
no shutdown
!
interface Port-channel 128
mtu 12000
portmode hybrid
switchport
fip-snooping port-mode fcf
no shutdown
link-bundle-monitor enable
!
Cisco Fiber Hub:
Port-channel/Etherchannel Config:
!
interface Port-channel128
switchport mode trunk
spanning-tree portfast trunk
!
Interface Configs (ports):
!
interface TenGigabitEthernet1/0/1
switchport mode trunk
channel-group 128 mode active
spanning-tree portfast trunk
!
interface TenGigabitEthernet1/0/2
switchport mode trunk
channel-group 128 mode active
spanning-tree portfast trunk
!
interface TenGigabitEthernet1/0/3
switchport mode trunk
channel-group 128 mode active
spanning-tree portfast trunk
!
interface TenGigabitEthernet1/0/4
switchport mode trunk
channel-group 128 mode active
spanning-tree portfast trunk
!
Issue:
I can see all of these connections in vSphere under the Physical Adapters for all 4 of the hosts. For ease of explanation, lets say all 8 connections from the chassis (4 from each FN410s IOM) are vmnic0 through vmnic7.
vmnic0, 2, 4, and 6 belong to 'A1' (top IOM) and vmnic1, 3, 5, and 7 belong to 'A2' (bottom IOM).
Monitoring the Cisco Discovery Protocol (CDP) tab on each of these physical connections within vSphere, all physical adapters on 'A1' (vmnic0, 2, 4, 6) connected to the Cisco Fiber Hub change every few seconds, rotating through Te1/0/1, Te1/0/2, Te1/0/3, and Te1/0/4. The same goes for all physical connections on 'A2' (vmnic1, 3, 5, 7). This part, I am not sure is an actual issue since each of the 4 physical/external connections are in LACP as 1 logical connection. I would think that the physical connections would still display what they are physically connected to and not cycle through each of the 4 ports on the Cisco Fiber Hubs.
I created the virtual distribution switch in vSphere and assigned each of the 4 ESXi Hosts to this switch. All configurations on this switch have been verified and confirmed, including VLAN tagging, etc. When I swap network adapter connections on each of my VMs on these hosts from the old network adapter connection the this new one I have created, about half of the VMs connect to the network correctly and the other half do not. Those that are not connecting cannot be pinged from another device.
At this point, I do not know where the issue stems from, but I believe it to be a configuration issue on the FN410s IOMs. Any advice or assistance is appreciated.
P.S. I am currently reading through this manual to see if anything sticks out to me: https://dl.dell.com/topicspdf/poweredge-fx2_reference-guide8_en-us.pdf
I have gone through just about every documentation on FN410s modules I can search for but am just lost on how to configure these modules.
Thanks.
DELL-Charles R
Moderator
•
4.4K Posts
0
November 8th, 2022 11:00
Hello zbg2021,
Please note we are unable to assist directly in initial configurations, we can leave it open for community involvement.
We do have professional services which can be contracted to deploy the configuration you need. You may consider contacting Dell Sales department for Prodeploy Service if needed.
zbg2021
5 Posts
0
November 14th, 2022 07:00
I was able to configure the switches in Full Switch mode and they are working as I intended originally.
The issue that I was experiencing had to do with the utilization of the the port-channel/LAG/LACP within VMware and on the FN410s Modules.
Now, they are basically layer 2 switches and just pass VLAN traffic like a normal switch that is connected to each of the blades internally on interfaces te0/1, 3, 5, and 7 with no port-channel in use. In most configurations for these IOA modules, they are only connecting to 1 Top-of-Rack switch. In my scenario, I have 2 ToR switches. IOA A is used as primary to ToR switch 1, and IOA B is used as redundant to ToR switch 2 . Since both IOAs are not connecting to the same ToR switch, I cannot utilize the primary/redundant connection with port-channels. The only redundancy would be if 1 of the 4 connections from 1 IOA went down, the other 3 would still be up. I am/was looking for redundancy on a ToR switch level, not from the IOA level.
My issue now is that I need to be able to utilize VLAN 1 as a trunked VLAN, but the current configuration is that VLAN 1 is the Management VLAN and interfaces cannot be tagged (trunked) to VLAN 1. I have found information from another community post that I think I might be able to utilize: https://www.dell.com/community/Networking-General/Dell-FX2-FN-410S-IO-Aggregator-mgmt-issue-when-vlan-1-is-allowed/td-p/4670018