Unsolved
This post is more than 5 years old
1 Rookie
•
16 Posts
0
16311
February 10th, 2011 12:00
MD3200i Problem
I have a MD3200i with two R610 servers and two PC6224 Network Switches.
The problem I am having the two servers operate VERY slowly over ethernet. There are four subnets for each card.
192.168.130.X/24
192.168.131.X/24
192.168.132.X/24
192.168.133.X/24
Both VMWare servers are connected using the Broadcom 5709 network adapters (not configured thru BIOS). When I have high reads/writes, vmware says that the connect is lost, then immediately reset itself. I have talk with VMWare and they said it's a problem with the MD3200i's. They ran a program called "esxtop", then press d, and I had a extemly high iSCSI command delay.
Anybody know what I could be doing wrong?
Dev Mgr
4 Operator
•
9.3K Posts
0
February 10th, 2011 18:00
Can you paste in the output of:
From the server (SSH):
- esxcfg-vswitch -l
- esxcfg-vmknic -l
- esxcli swiscsi nic list -d vmhba## (where ## is the instance number of the software iSCSI initiator (assuming that you're running the software iSCSI initiator)
From the 6224 switches:
- show running-config
lrnajjones
1 Rookie
•
16 Posts
0
February 10th, 2011 19:00
~ # esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 128 12 128 1500 vmnic4
PortGroup Name VLAN ID Used Ports Uplinks
Commission Network 0 9 vmnic4
Management Network 0 1 vmnic4
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch2 128 9 128 1500 vmnic0,vmnic1,vmnic2,vmnic3
PortGroup Name VLAN ID Used Ports Uplinks
iSCSI4 0 1 vmnic3
iSCSI3 0 1 vmnic2
iSCSI2 0 1 vmnic1
iSCSI1 0 1 vmnic0
~ # esxcfg-vmknic -l
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk0 Management Network IPv4 172.20.20.72 255.255.254.0 172.20.21.255 f0:4d:a2:06:58:1c 1500 65535 true STATIC
vmk0 Management Network IPv6 fe80::f24d:a2ff:fe06:581c 64 f0:4d:a2:06:58:1c 1500 65535 true STATIC, PREFERRED
vmk2 iSCSI1 IPv4 192.168.130.13 255.255.255.0 192.168.130.255 00:50:56:7d:b5:0d 1500 65535 true STATIC
vmk2 iSCSI1 IPv6 fe80::250:56ff:fe7d:b50d 64 00:50:56:7d:b5:0d 1500 65535 true STATIC, PREFERRED
vmk3 iSCSI2 IPv4 192.168.131.13 255.255.255.0 192.168.131.255 00:50:56:7e:0f:9b 1500 65535 true STATIC
vmk3 iSCSI2 IPv6 fe80::250:56ff:fe7e:f9b 64 00:50:56:7e:0f:9b 1500 65535 true STATIC, PREFERRED
vmk4 iSCSI3 IPv4 192.168.132.13 255.255.255.0 192.168.132.255 00:50:56:70:03:ae 1500 65535 true STATIC
vmk4 iSCSI3 IPv6 fe80::250:56ff:fe70:3ae 64 00:50:56:70:03:ae 1500 65535 true STATIC, PREFERRED
vmk5 iSCSI4 IPv4 192.168.133.13 255.255.255.0 192.168.133.255 00:50:56:79:08:a9 1500 65535 true STATIC
vmk5 iSCSI4 IPv6 fe80::250:56ff:fe79:8a9 64 00:50:56:79:08:a9 1500 65535 true STATIC, PREFERRED
~ #
vmware2-switch>enable
vmware2-switch#show running-config
!Current Configuration:
!System Description "Powerconnect 6224, 3.2.0.7, VxWorks 6.5"
!System Software Version 3.2.0.7
!Cut-through mode is configured as disabled
!
configure
vlan database
vlan 100,200-203
vlan routing 200 1
vlan routing 201 2
vlan routing 202 3
vlan routing 203 4
exit
hostname "vmware2-switch"
sntp unicast client enable
sntp server 172.20.20.12
clock summer-time recurring USA zone "CDT"
clock timezone -6 minutes 0 zone "CST"
stack
member 1 1
member 2 1
--More-- or (q)uit
exit
ip address 172.20.20.74 255.255.254.0
ip default-gateway 172.20.20.1
ip name-server 172.20.20.12
ip name-server 172.20.20.9
logging file debug
logging buffered debug
ip routing
interface vlan 200
name "iSCSI0"
routing
ip address 192.168.130.1 255.255.255.0
exit
interface vlan 201
name "iSCSI1"
routing
ip address 192.168.131.1 255.255.255.0
exit
interface vlan 202
name "iSCSI2"
routing
--More-- or (q)uit
ip address 192.168.132.1 255.255.255.0
exit
interface vlan 203
name "iSCSI3"
routing
ip address 192.168.133.1 255.255.255.0
exit
username "admin" password ************* level 15 encrypted
!
interface ethernet 1/g5
description 'MD3200i Array 2 RC0 P0'
spanning-tree disable
switchport access vlan 200
exit
!
interface ethernet 1/g6
description 'MD3200i Array 2 RC0 P1'
spanning-tree disable
switchport access vlan 201
exit
!
interface ethernet 1/g7
description 'MD3200i Array 2 RC0 P2'
spanning-tree disable
switchport access vlan 202
exit
!
interface ethernet 1/g8
description 'MD3200i Array 2 RC0 P3'
spanning-tree disable
switchport access vlan 203
exit
!
interface ethernet 1/g11
description 'VMSERVER2 iSCSI0'
spanning-tree disable
switchport access vlan 200
exit
!
interface ethernet 1/g12
description 'VMSERVER2 iSCSI1'
spanning-tree disable
switchport access vlan 201
exit
!
interface ethernet 1/g15
description 'Symantec BackupExec Server'
spanning-tree disable
switchport access vlan 202
exit
!
interface ethernet 1/g19
description 'APC-ATS Switch'
exit
!
interface ethernet 1/g24
description 'Uplink'
switchport mode general
switchport general allowed vlan add 100 tagged
exit
!
interface ethernet 2/g5
description 'MD3200i Array 2 RC1 P0'
spanning-tree disable
switchport access vlan 200
exit
!
interface ethernet 2/g6
description 'MD3200i Array 2 RC1 P1'
spanning-tree disable
switchport access vlan 201
exit
!
interface ethernet 2/g7
description 'MD3200i Array 2 RC1 P2'
spanning-tree disable
switchport access vlan 202
exit
!
interface ethernet 2/g8
description 'MD3200i Array 2 RC1 P3'
spanning-tree disable
switchport access vlan 203
exit
!
interface ethernet 2/g11
description 'VMSERVER2 iSCSI2'
spanning-tree disable
switchport access vlan 202
exit
!
interface ethernet 2/g12
description 'VMSERVER2 iSCSI3'
spanning-tree disable
switchport access vlan 203
exit
!
interface ethernet 2/g19
switchport access vlan 100
exit
snmp-server community public ro
snmp-server host 172.20.20.2 public informs
exit
vmware2-switch#exit
vmware2-switch>quit
Dev Mgr
4 Operator
•
9.3K Posts
0
February 11th, 2011 10:00
Are you trying to use the Broadcom hardware iSCSI offload? You didn't include the "esxcli swiscsi nic list -d vmhba##" output for the software iscsi initiator instance.
Per these 2 links, it seems that some testing shows that hardware iSCSI (on Broadcom 1Gbit NICs) without jumbo frames is significantly slower than software iSCSI with jumbo frames:
- http://www.delltechcenter.com/thread/4100544/vSphere+4.1+upgrade
- http://www.vmadmin.co.uk/vmware/35-esxserver/252-esxihwswiscsijumbo
I'd suggest the following changes:
On the switches:
- for each iSCSI port, use the following settings: "mtu 9216", "spanning-tree portfast", "storm-control broadcast", "storm-control multicast"
On the servers:
- rebuild your iSCSI setup using the software initiator with jumbo frames. You can use http://www.delltechcenter.com/page/VMware+ESX+4.0+and+PowerVault+MD3000i to find the commands to do this.
After that, you'll want to see how your setup ends up running.