This post is more than 5 years old

1 Rookie

 • 

31 Posts

2079

June 25th, 2019 13:00

volume growing every day - solution?

equallogic PS6100.

have two of them, one syncreps volumes to the other.

i have this one thin provisioned volume that I store some linux servers on.

this volume is growing by 3% every day for the last few days, up to  to 77% in use space now.

although I could just add more space to the volume ( I have plenty free), doesnt change the fact that it continues to grow.

not 100% clear on what causes the growth, but I think Eql doesnt allow you to reclaim space after it writes to block?

I have 13 VMs stored on this volume, total HD space for all of them is 904 GB.

Vmware shows 3 TB Datastore, 989 provisioned, 2 TB free.

EQL shows in-use 2.31TB, with 7.2 unreserved.

unless I change something, it will just continue to grow, so what are my options?

change volume so it is no longer thin -provisioned?

move everything to another volume?

 

4 Operator

 • 

1.5K Posts

June 25th, 2019 21:00

Hello,

 SANs do not reclaim on their own.  They can't they aren't filesystem aware.  The owner of the volume is responsible for telling the storage what blocks can be freed.   VMFS v5+ supports reclaim. AKA UNMAP

This VMware KB covers how to do that.

https://kb.vmware.com/s/article/2057513

 On the EQL side you need FW greater than 6.x. The volume cannot be replicated by EQL using SYNC or ASYNC replication.   UNMAP is disabled on replicated volumes on Dell PS Series SANs.

 But to remove your concern overall.  Nothing bad will happen if you use 100% of an EQL volume.  In the case of thin provisioning this assumes you have not overprovisioned your actual storage capacity.  I.e. you have 10TB of actual space and create a 15TB thin provisioned volume.  Once you run out of physical space, the volume goes offline.

 So let's assume you have not done that.  As I mentioned SANs don't know filesystems or files.  A server tells the SAN to write or read a block of data.  If a delete occurs and no corresponding UNMAP command is sent there's no notification to the SAN to free that block.

 The OS and filesystem keep track of all the used and free blocks on each volume  That allocation table allows the OS to re-use previous blocks.

 So even if an EQL volume shows 100% in-use, the OS could show 100% FREE.  If you filed the volume and then deleted everything.  So do not use the EQL GUI as an up to date gauge of free and used space.

 So when you get to the threshold on EQL side, you do not have to increase the volume.  You can also change the warning alert all the way to 100% if you have enough space to hold all the volumes data.  Again, NOT over provisioned your actual disk space.

 Now a funny thing on how OS's vs. SANs are different.  When you first create a VM on a 100% free EQL volume an interesting thing will happen.  With no thin provisioned VMDKs the array will show MORE free space than VMware reports.  I.e. if you create a 500GB VMDK thick provisioned hardly any space will show used on the EQL side. But 500GB will show used on ESXi side.  That's because VMware doesn't actually write out 500GB when it creates a VMDK file.  Far from it.   Over time it will swing the other way as VMs are deleted or moved to another volume.  So MORE space will show allocated in EQL GUI vs. VMware. 

 Also until you get to ESXi v6.7 U2 or greater, and native VMFS v6 formatted volumes, when you delete a file inside a VM UNMAP commands from that VM's OS are not passed to the backend SAN storage.

 Which means only when ESXi deletes a file will space be reclaimed when using the VMFS reclaim process.

So temp files, VMDK snapshots, log files, memory dump files, deleted or moved VMs will be reclaimed.

So bottom line.  Don't go by what EQL shows for in-use space vs. free on a volume.  Go by VMware info.

 Regards,

Don

 

 

 

4 Operator

 • 

1.5K Posts

June 26th, 2019 04:00

Hello,

 You are very welcome.  I am glad that I could assist you.

 All block storage devices work the same way.

 What's different is the PAGE size they are using.  That can be anything from 512b, 1M, 2M, 8M, 15M, etc...

 Which means a single write to that page will allocate whatever the page size is.  It's 15MB on PS Series SANs.

 But any additional writes to that page don't use any more space.  Just fill up that page.

 So in-use space graphs can grow fast in environments with lot's of write.  Also, almost all filesystems by default update the last access date/time on files and directories.  So reads also involve a write. :(

 On the EQL support site with the firmware downloads is a PDF that shows how to disable that.  it's called the OS Considerations guide.

 Along with setting DiskTimeOut values that all VMs should have.

 Since you are running VMware you should also check that all the best practices for that are set as well.

https://downloads.dell.com/solutions/storage-solution-resources/BestPracticesWithPSseries-VMware%28TR1091%29.pdf

 Regards,

Don

1 Rookie

 • 

31 Posts

June 26th, 2019 04:00

thanks, that really helped me understand how eql and vmware see space differently.

 

No Events found!

Top