Unsolved
This post is more than 5 years old
2 Intern
•
294 Posts
0
54107
February 26th, 2013 07:00
Broadcom/Equallogic Questions...
Hi,
I was trying to email Dell tech support, but the button redirects me back to the tech support site...
I have an open case (SR [Note: Number Removed per Dell Privacy Policy] ) and the Equallogic support person told me to contact Dell server support to find out how to configure the Broadcom NICs.
Anyways, this is what the EQL support person told me to do and I do not know how to do this:
-Disable Offloading Technologies on All iSCSI Network Adapters
-Disable TOE in Windows
-Disable TOE in Broadcom Advanced Control Suite
-Disable Receive Side Scaling
-Disable Large Send Offload
-Enable Flow Control on all iSCSI NIC Adapters
-Disable TCP Chimney
-Disable Windows 2008 R2 Auto Tuning
Some background information regarding the way the servers are connected to the EQLs:
- iSCSI traffic is separate from the public NIC using the Broadcom NICs. They are using the Broadcom adapters (Broadcom Advanced Control Suite 4) that controls the four NICs. They look disabled to Windows because that's how the Broadcom drivers work. Also, Dell Equallogic MPIO tab in iSCSI Initiator excludes the public NIC IP Address.
- In the iSCSI Initiator Discovery tab, the IP Addresses listed are the Broadcom NICs so that is correctly configured.
EQL support: Is there some kind of teaming done with those nic’s. We need those 4 nics to be separately connected and we should see each establishing session independently to Equallogic . we are aware that the MPIO table has the exclusion but the physical nic seen is only one . if the other nics are enabled why don’t we see them when ipconfig is done
Answer: There is no teaming. Each of the four NICs has its own IP on the subnet. The NICs are configured using the Broadcom software, which hides them from Windows for some reason presumably having to do with its iSCSI offload implementation. I don't know the details of how or why the Broadcom software does what it does, but you can verify that each one is separately connected on both the Dell EqualLogic MPIO tab in iSCSI Initiator and the Monitoring > Statistics > iSCSI Connections section of the EqualLogic array console.
Phew!
Can someone please tell me how to do this? Do I need to unconfigure the Broadcom network adapters first?
Or, can someone from the Dell Server Tech Support group email me?
Thank you.
DELL-Joe S
7 Technologist
•
729 Posts
0
February 26th, 2013 08:00
See attached.
-joe
1 Attachment
broadcom_nic_config_guidelines.pdf
dajonx
2 Intern
•
294 Posts
0
February 26th, 2013 08:00
Thanks, Joe.
I've looked at the attached pdf, but still have questions...
When looking at BACS in the configurations tab under the Resource Reservations tree, I see that NDIS and iSCSI Offload Engine are both enabled and TOE is disabled in Pre-Allocated Resources. I do not see the Receive Side Scaling and Large Send Offload options in BACS. Also, the Advanced tree are all set to defaults (Flow Control: Auto, Pause On Exhausted Host Ring: Disabled, Speed & Duplex: Auto Negotiation). Is that accurate?
-So I do not have to unconfigure the Broadcom network adapters before changing the settings?
-How do I disable TOE in Windows 2008 R2?
-All of the other config changes is done in BACS?
-In BACS, I highlight the Broadcom adapters, click on the configurations tab, and change the settings in the Resource Reservations tree, correct?
Thank you.
DELL-Joe S
7 Technologist
•
729 Posts
0
February 26th, 2013 10:00
-I do not see the Receive Side Scaling and Large Send Offload options in BACS. Also, the Advanced tree are all set to defaults (Flow Control: Auto, Pause On Exhausted Host Ring: Disabled, Speed & Duplex: Auto Negotiation). Is that accurate?
A: this is only available if NDIS mode with LSO and Checksum Offload functions are DISABLED on the adapter
-So I do not have to un-configure the Broadcom network adapters before changing the settings?
A: Correct, but once you save (apply) the changes, the adapter will be restarted. Ensure you have your iSCSI disk timeout values setup properly on the Host(see the support site: downloads> ps series firmware> select the FW installed on your array(s)> click the link next to download page> in the documentation section review the file titled: “iSCSI Initiator and Operating System Considerations”
-How do I disable TOE in Windows 2008 R2?
A: If TOE is not select in BACS, it’s disable. Also, ensure the TCP Chimney Offload is not setup. The document has a link for Micorsoft settings for this in the section titled “TOE mode configuration:”
-All of the other config changes are done in BACS?
A: Yes
-In BACS, I highlight the Broadcom adapters, click on the configurations tab, and change the settings in the Resource Reservations tree, correct?
A: Yes
The important thing here is that there are three modes to choose from, (iSCSI Offload mode, TOE mode and NDIS mode). So depending on your needs, you should try each to see which one gives you the best performance, and server CPU utilization.
-joe
dajonx
2 Intern
•
294 Posts
0
February 26th, 2013 11:00
Thank you, Joe. That was extremely helpful!
I have asked the EQL support guy which mode I should have enabled (iSCSI Offload, TOE, or NDIS). He said to disable all offloading technologies so that would mean to only enable NDIS which will then mean that RSS/LSO options cannot be disabled, correct?
DELL-Joe S
7 Technologist
•
729 Posts
0
February 26th, 2013 13:00
According to the document, you would use the following settings for NDIS:
From the Device Management screen by Connection, Configurations tab, select NDIS
Configuration Tab:
Flow Control = Tx Enabled or Rx & Tx Enabled
Interrupt Moderation = Enable *see note 1
Large Send Offload = Disable
Checksum Offload = None
Receive Side Scaling (RSS) = Enabled
Jumbo MTU = enabled when the value of this field is 9000
TCP Connection Offload = Disable Wake Up Capabilities = None
-joe
dajonx
2 Intern
•
294 Posts
0
February 27th, 2013 06:00
Thank you, Joe.
Just to be absolutely clear that I'm looking at the correct windows, I go to Control Panel/Network and Internet/Network Connections and select the disabled Broadcom NICs. Right-click on them, select Configure, go to Advanced tab and change the following properties:
-Flow Control (currently Rx & Tx Enabled)
-Interrupt Moderation (currently Enabled)
-Large Send Offload V2 (IPv4) (currently Enabled - needs to be Disabled)
-Large Send Offload V2 (IPv6) (currently Enabled - needs to be Disabled)
-TCP/UDP Checksum Offload (IPv4) (currently Rx & Tx Enabled - needs to be Disabled)
-TCP/UDP Checksum Offload (IPv6) (currently Rx & Tx Enabled - needs to be Disabled)
-Receive Side Scaling (currently Enabled - needs to be Disabled)
-Jumbo Packet (currently 1514 - needs to be 9000)
-TCP Connection Offload (IPv4) (currently Disabled)
-TCP Connection Offload (IPv6) (currently Disabled)
As for the Registry Editor changes per "iSCSI Initiator and Operating Systems Considerations" whitepaper, could you please tell me if these values are good? (I've never done this before and want to double-check)
LinkDownTime - REG_DWORD - 0x00000023 (35)
TimeOutValue - REG_DWORD - 0x0000003c (60)
UseCustomPathRecoveryInterval - REG_DWORD - 0x00000001 (1)
PDORemovePeriod - REG_DWORD - 0x00000078 (120)
PathRecoveryInterval - REG_DWORD - 0x0000003c (60)
As for finding the LinkDownTime parameter, do I just search through the 0000-0007 folders looking for the property? Or is it supposed to be in a certain folder?
I really appreciate your help, Joe. I wish I had you as my POC for my EQL issues.
dajonx
2 Intern
•
294 Posts
0
February 28th, 2013 05:00
Can someone please look over my previous post? I would like to make these changes later today.
dajonx
2 Intern
•
294 Posts
0
February 28th, 2013 07:00
I apologize for interrupting your vacation, Joe. Thank you for taking the time to respond.
Those registry entries are from the "iSCSI Initiator and Operating System Considerations" document for my OS and firmware.
Again, thank you for your help, Joe. Have a great time during your time-off!
DELL-Joe S
7 Technologist
•
729 Posts
0
February 28th, 2013 07:00
Sorry Dajonx... but I'm on vacation and will not be back until the March 4th. The changes you listed for the broadcom look correct.
Regarding the registry entries, these are per OS and PER firmware, provided you consulted the "iSCSI Initiator and Operating System Considerations" document, and these are the ones for your OS, then they are good. You also will need to restart the host server for any of the changes to take effect.
-joe