This post is more than 5 years old
1 Rookie
•
32 Posts
0
1745
July 22nd, 2015 08:00
Different Frame Size For Subnets On The Same Interface
Hello,
I have a customer with a 3-node Isilon cluster. It has two pools on different subnets on the same network interfaces. "Client_Subnet" is uses for Windows CIFS Clients. "NFS_Subnet" is used for vSphere NFS Datastores. Both subnets use the same 10gigE interfaces. Cluster is on OneFS 7.1.1
Subnet #1
Name: Client_Subnet
Subnet: 192.168.1.0
Netmask: 255.255.255.0
MTU: 1500
Gateway 192.168.1.254
with pool
Client_Pool
IP-Range: 192.168.1.11 - 192.168.1.13
Access Zone: System
Pool Members: 10gige-agg1 (Node 1-3)
Subnet #2
Name: NFS_Subnet
Subnet: 192.168.2.0
Netmask: 255.255.255.0
MTU: 1500
with pool
Client_Pool
IP-Range: 192.168.2.11 - 192.168.2.13
Access Zone: System
Pool Members: 10gige-agg1 (Node 1-3)
The customer wants to use jumbo frames on the NFS subnet. Regarding to Isilon OneFS CLI Administration Guide MTU can be modified on a subnet base using "isi networks modify".
When trying to set the MTU for "NFS_Subnet" to 9000 I get the following errors:
isilon1-rv-1# isi networks modify subnet --name=NFS_Subnet --mtu=9000
Modifying subnet 'NFS_Subnet':
!! modify subnet command failed: The following interface would be
!! assigned to subnets with conflicting MTUs: 1:10gige-agg-1. Use the
!! --force option to override this warning. Configuration will not be
!! saved.
How is ist possible to run two different subnets on the same interfaces, one running with MTU 9000 and the other with MTU 1500?
Best regards
Robert Spiess
0 events found
crklosterman
450 Posts
1
July 22nd, 2015 08:00
Robert,
The short answer is that this is not possible. A frame is an OSI layer-2 concept. At layer-2 you can only have one MTU for the physical links, not 2. That said TCP negotiates window sizes (higher-layer), so if half the clients have 1500MTU and the cluster uses 9K to match the upper limits for some subset of clients, that’s OK, it should negotiate down per session. Ensure the cluster is on current code and firmware first there have been a few bugs when jumbo frames are set that have been fixed, but you don’t want to run into those.
So how do you get all of this to 9K since you can only change 1 subnet at a time?
You’ll have to pull all the interfaces out of one of the subnets temporarily, or at least ensure there is no overlap between them. Meaning maybe SMB on nodes 1 and 2 temporarily, and NFS on node 3 temporarily. Then change 1 subnet. Then change the other, then put all the interfaces back where they were to begin with. Will this cause client interruptions? Yes. The impact can however be minimized if done correctly by suspending nodes from pools in advance. Read up on --suspend-nodes in the OneFS Command Reference documentation. I can try and explain it further if you have more questions.
Chris Klosterman
Email: chris.klosterman@emc.com
Advisory Solution Architect
Offer and Enablement Team
EMC²| Emerging Technologies Division
r_spiess
1 Rookie
•
32 Posts
0
August 3rd, 2015 03:00
Thanks.
Robert