Unsolved
This post is more than 5 years old
4 Posts
0
6889
September 25th, 2009 14:00
Celerra network configuration best practices
Just wanted to solicit feedback on a couple different approaches to networking on the Celerra (Specifically NS120 which is what our client base is adopting at the greatest rate)
System is configured with quad copper gig nics. Client needs access to both CIFS and iSCSI via this copper links.Assume usage of Cisco switching gear as the network core. In most cases, single switch except where noted below.
Scenario A:
CGE0, and CGE2 configured as LACP trunk named cifs_trunk
CGE1 and CGE3 configured as LACP trunk named iscsi_trunk
Create an iSCSI target device with IP 10.10.10.10 for example.
Create another device for use with CIFS server1 with IP 10.100.10.10 for example.
Question here is 1) Is etherchannel or LACP preferred? Why? 2) Does this satisfy both load balancing AND failover?
Scenario B:
Device is configured using CGE0 with IP 10.10.10.10
Device is configured using CGE2 with IP 10.10.20.10
CGE1, and CGE3 is configured as LACP trunk named cifs_trunk
Device is created for use with CIFS server1 with IP 10.100.10.10
Question here is 1) Does this provide true multipathing whereas Scenario A doesn't? Is the end result the same, just different approach? 2) Is there a better configuration option for CIFS?
Scenario C: Assume dual switches
CGE0 and CGE2 are configured as FSN instead
What does everyone else do? What have I left out? Just curious what other engineers are seeing out there.
DannoOfNashvill
6 Posts
0
September 29th, 2009 12:00
For your first scenario, whether you choose LACP or Etherchannel depends on what's important to you. Etherchannel will provide much faster failover on link failure/restore - On the order of 50 milliseconds. LACP failover/restore will depend on the timers you set. For most switches, you can configure short timers and get it down to about 1.5 seconds in either direction.
In both cases in the first scenario, you'll be successfully using both legs of an LACP, and it'll be roughly equal (given a fair number of hosts).
Of course, the switch will be a single point of failure. You could either
a) Connect one LACP group to one switch, one LACP group to another switch, and run both iSCSI and CIFS over each. You can use 802.1q if you like.
b) Use switching gear that'll allow you to split a multi-link trunk over multiple physical switches. Nortel, which is now Avaya, does this on quite a few models (5500, 5600, 8300, 8600). Cisco does this on some of their high end gear (6500s w/ 10GbE and Sup720).
Note that in (b) above, you could turn on 802.1q and aggregate all four links. You'd have a 4Gb link with hardware multipathing that everyone was sharing. Pretty nifty.
In your second scenario - I'm not sure I quite get that one.
For the third scenario - See my response to the first scenario above.
ed0g1
4 Posts
0
September 29th, 2009 12:00
Thanks for the feedback.
The goal with Scenario B is to use a more "AX"-centric approach to failover. In that case you are looking at two seperate subnets, so hosts with multiple ports can use iSCSI on each of the two subnets. But if scenario A provides the same level of multipathing all the way down to the host, then it would appear that I would get the greatest benefit from using LACP and eliminating some single points of failure.
The idea of doing an LACP aggregration for ALL four copper ports also intrigues me. Never thought of taking this approach. So using my earlier examples, you can exapand to a couple of choices if I am thinking of this correct.
Scenario D:
CGE0, 1, 2, 3 are all configured as part of a LACP trunk and configured as device celerra_trk
A network interface is created for iSCSI on the celerra_trk device with IP 10.10.10.10
A network interface is created for CIFS on the same celerra_trk device with IP 10.100.10.10
All traffic is then shared amongst the four copper ports on the Celerra
Scenrio E: (Assuming two switches)
CGE0, and CGE2 are configured as an FSN on the two switches as lacpA_trk
CGE1, and CGE3 are also configured as an FSN on the two switches as lacpB_trk
The two FSN devices are then bonded as etherchannel and the same interfaces are created as above.
Difference here is that you now have end to end protection and redundancy.
DannoOfNashvill
6 Posts
0
September 29th, 2009 13:00
Okay, that makes sense. Something akin to what you describe in B may be best if there turn out to be any issues with LACP or trunking in your particular environment.
Yup. Scenario E would be easiest if you're hooking into two basic switches (from Cisco, Nortel, 3com, Extreme, anybody). I think it'd do a great job. Scenario D is great if you’ve got hardware that can do it.
Before I came to work for EMC, I spent quite a bit of time testing essentially those two scenarios with a small Clariion for a customer on networking gear from both Cisco and Nortel (they had a multi-vendor networking environment). The customer was very serious about redundancy, so I went through about every scenario you could imagine.
The result of the LACP (or Etherchannel) trunking: Performance pretty much scales linearly as long as you can add links, so long as there's not some other bottleneck. When we tested with 4x 1gig links, we got four times the performance of a single link (minus a couple percent). LACP was picked over Etherchannel because it was more resilient against cabling errors - The LACP protocol has end-to-end error-checking, meaning if some jerk moves a cable to another switch or port, you won't get black-holed frames.
The other result of the testing - you get a few percentage points of benefit by completely turning off the QoS buffers on the storage ports. However, that benefit only holds if you're absolutely sure you've got no backplane contention. If you're not totally confident, set your buffers as large as they'll go, and you'll get the most reliable performance.