What is T10 UNMAP?
UNMAP is a SCSI command used to reclaim space from blocks that have been deleted by a virtual machine (OS or application).
In vSphere 5, UNMAP is used for space reclamation of deleted data after common operations
This is particularly beneficial and important in thinly provisioned environments as it allows the storage array to realise these are unwanted or unused blocks and to return them to the free capacity pool.
What makes UNMAP important?
HP 3PAR thrives on the thin suite and it supports UNMAP as of Inform OS 3.1.1, however the first release of vSphere 5.0 had some issues where there were unexpected timeouts when the UNMAP command was issued from the ESXi host during an operation.
So when an operation like storage v-motion or a virtual machine is deleted, a copy or movement of data is kicked off essentially leaving behind deleted blocks, HP 3PAR can only realise this so long as UNMAP is started to reclaim those blocks. Since administrators are using storage v-motion on a day-to-day basis, the impact can be huge.
So what do you do?
Disable it until VMware release a fix – expected in the next patch release.
Use a manual command such as sdelete on Windows or dd on Linux to write zeros at the file system level, The 3PAR’s ASICs will pick these zeros up so long as zero detect is enabled.
How can UNMAP be disabled?
Support for UNMAP in our storage arrays is enabled by default and cannot be disabled by customers. In vSphere support for UNMAP is enabled by default but can be disabled via the command line interface.
esxcfg-advcfg –s 1 /VMFS3/EnableBlockDelete
This can be completed automatically by installing ESXi 5.0 Patch 02. For more information, see VMware ESXi 5.0 Patch Image Profile ESXi-5.0.0-20111204001-standard (2009330).
Summary
- This issue only affects thin provisioned arrays in the 3PAR family
-
UNMAP is a SCSI command standardized within T10 SCSI command set – It is not specifically a vSphere 5 feature
- This Issue only occurs when using ESXi 5.0 and 3PAR arrays that have the latest firmware.
- Customers can still reclaim space without UNMAP using the 3PAR arrays zero detect functionality should they disable UNMAP
- A patch is available that resolves this issue
This is excellent information and I am sure will be appreciated by many. Well done for writing this!
Thanks Shane, more to come in the 3PAR\VMWare space!
Hi,
We have 3 ESXi running 5U1… We have too a 3PAR.
We have a Windows 2008 machine with 10 RDMs.
We have tried to convert our RDMs (Thick Eager) to virtual disks and the processsus failed…
Error : imcompatible device backing specified 0
Now we have tried one after one and the processus goes ok but with very very bad performance…
Any ideas ?
We appreciate your help.
Hi,
What process are you using to convert the RDMs?
Also what are the RDM luns hosting (applications)
Andre
You state that the issue is resolved in ESXi 5.0 Patch 02. While actually the only thing that happened, is that the UNMAP feature is disabled by default, that is not a fix! The UNMAP feature not working means alot of babysitting your CPG`s, since after every sVmotion or deletion of a VMDK, the space will not be reclaimed automaticly, you need to zero out the space manually by running the dd command on your ESX host, or by just filling up the free space of a datastore with a thick eager zeroed vmdk and deleting it.
Hi Arris,
Bad choice of wording on my behalf, I had performed some testing in one of my labs and managed to get UNMAP to working with our 3PAR – ESXi 5 host was running patch 02 .
Understand the implications of reclaiming manually, was hoping since 4.x there would be a better solution.
Andre
Hiya,
I’ve created a powershell script to run through and create large blocks of zeros on each selected datastore to complete this process (not ‘pretty’ but works VERY effectively). I’ve just uploaded this to the VMware community forum: http://communities.vmware.com/docs/DOC-23670
Cheers
Steve
Thanks Steve – thats awesome stuff., we have just released a whitepaper on Thin Provisioning Best practices
http://h20195.www2.hp.com/v2/GetPDF%2Easpx%2F4AA3%2D8987ENW%2Epdf
Reclamation is covered on the various O/S include vSphere 5.0, support for automatic reclamation was pulled after some performance problems in some environments. vmkfstools can be used in any event…
Hiya Andre,
We looked at vmkfstools but couldn’t see a way to automate it across all 100+ datastores presented in each cluster hence the (ridiculously large) script that was conceived.
Would be interested to note if anyone has used vmkfstools in such a manner (we could find no references) or whether others are using cron jobs or such to run this on a regular basis.
Steve
Hi Andre
How quickly would you expect to see the space reclaimed on the 3PAR after creating the EZT Disks. I have been creating EZT disks and expecting a near instant reduction in used space, as we have with our XIV, is that the case with the 3PAR?
thanks in advance
Chris