Celerra NX-4 - vSphere 4.1 NFS - problems

Robert,

NFS is and has been ready for primetime.  In order to help you out here it would be nice to have some more information about the workload, amount of VMs, and if there are any tools you have used to becnhmark the connectivity from your ESX servers to the NFS exports.

The stuttering actually rings out to me as an issue possibly with "all paths down."  Depending on the exact version of ESX you are running, if you remove block datastores (iSCSI/FC) without deleting them form the ESX server, the ESX/i host does not respond well to the missing LUN and can cause rather substantial problems that can be seen as stuttering/loss of network connectivity to the host.  If you're willing to give it another try, one option is to reinstall ESX/i on a host and attach the fresh datastores to do your performance testing to ensure the backend disk is going to meet your needs.

There are also best practices around how fileysystems are mounted for NFS such as "directed writes (uncached writes) and prefetch disabled," available from advanced options under the NFS mount area.  These two options can help substantially in getting performance for NFS closer to the native block (iSCSI/FC/FCoE).

i removed all iscsi luns and paths when i built out the nfs datastores i used the vspere addon which was suppose to build out the nfs moints with best practices....

as far as load and bench marks. my load hasnt changed and iscsi handels it just fine...

could ibspend countless hours trying to shoehorn nfs in, sure. but i can tell u with in 20 mins of moving my vms to iscsi datastotes. users reported it was night and day... frankly thats the only benchmark that matters

ive been down this road twice now. nfs is great blah blah blah

i can tell u from my experience and countless hours on a celerra nx 4. its not

Sent from my iPhone

Robert,

The plugin does set one of the main best practices which is the "uncached" setting.  However, another one that might be useful for you would be the "prefetch" option which can be seen when you use the plugin to create nfs datastores under advanced.  Overall based on the NX4 load limits, the amount of data you are pushing to the NX4 shouldn't create a material difference for response times between the different storage protocols.  There is a lot of missing information here so it is very difficult to help resolve the issue.  A better place to get this resolved quickly if you want to use NFS would be EMC support or your partner.

If you have more information that you could provide as to the configuration and how the NX4 is laid out, the forum might be able to help a bit more in getting your issue resolved.

Thanks for sharing the informaiton.

I would also recommend reviewing the following blog post:

http://blog.scottlowe.org/2010/01/31/emc-celerra-optimizations-for-vmware-on-nfs/

It is a great summary, but for me is the start of a journey in that it has links to several whitepapers, and just as valuable, of course, are all the responses from others sharing their experiences (with additional links).

Clinton,

back in the days EMC recommendation was to set "uncached,noprefetch" for NFS file system to be used as vSphere stores. In the most recent documents like "Using EMC Celerra Storage with VMware vSphere and VMware Infrastructure" by Yossi Mesika and "VMware infrastructure deployment with EMC Celerra Unified Storage" you no longer see "noprefetch" recommendation.  Can you comment?

Thanks

The easiest way to address this is to say "it depends" =).  Prefetches have always been an imperfect way of trying to cache as much as possible on the read side.  With that said, a prefetch from the Celerra actually can yield more prefetching on the block side of the array as well, so the penalty if prefetching isn't providing value is much worse and can lead to spindle contention for no value.  This one is an easy change and can be set off and on from the advanced settings of a mount to "dial" the workload in for the best performance.  It all depends on the load..

what type of workload would benefit from using "noprefetch" ?

Highly random workloads which seem to be very common in VMware environment where you're mixing a lot of VMs together on datastores.  This would differ if you were to dedicate datastores for purposes, or if a VM in a datastore dominated the workload profile and makes that datastore prefetch friendly.  It all depends, but what it comes down to is that prefetch has never been a perfect science and isn't what you want in all situations and can be exaserpated by prefetching at file and blocks levels.  The NFS option is simply to stop the file based prefetch but will still have a block based prefetch in the backend.

Prefetch is in stark contrast to write cache which is almost always a good thing for a storage array and can lead to saving and minizming backend disk contention instead of adding to it for a % benefit.