Azure storage pools

You can combine multiple managed disks together in a storage pool to aggregate the size and performance of the disks which can give you a performance benefit for similar expense of a single disk of the same size. In addition as you generally need to go to larger disks to get a performance increase there is an opportunity to save costs by multiplying many cheaper and smaller disk together rather than choosing a single larger disk.

Based on a test using DISKSPD the IOPS and throughput values appear to sum together if using multiple disks, for example a single disk achieved a little over the 500 IOPS cap but when four disks are tested they achieve four times that at 2,121 IOPS and the same goes for throughput where the test pushed 2 MB/second of data with a single disk fours disks pushed 8 MB/second.

A storage pool uses simple (RAID 0) and does not support a parity setup (RAID 5) and in Azure each “disk” is actually stored in a BLOB in Azure Storage, which is replicated three times within the data center. It isn’t necessary to use any kind of disk resiliency.

Example difference between a single disk versus a pool

Using the PAYG model each managed disk offering has different size and performance options you can choose from to balance cost and performance requirements. The table below shows the Premium SSD disk offerings.

If you require a 4TiB disk you could choose a P50 which alone comes with 7500 IOPS and 250 MBps throughput. But by combining four P30 disks you achieve the same storage footprint of 4TiB but maximise the performance by combining the IOPS and throughput values togheter to achieve 20,000 IOPS and 800MBps.

1 x P50 4096GiB = 7,500 IOPS & 250MBps

4 x P30 (1024 GiB x 4) 4096GiB = (5000 IOPS x4) 20,000 IOPS & (200 MBps x 4) 800MBps

Example of performance increase for same cost

Storage required is 128 GiB but IOPS needs to burst beyond the 500 IOPS cap for end of day processing.

Example of cheaper costs using multiple smaller disks to achieve a higher IOPS

Storage required is 512 GiB but IOPS needs to run normally at higher rate than 500 IOPS.

Considerations and limits

If using storage pools for a SQL workload then consider it best practice to create seperate storage pools for the data and logs and not to create one big storage pool and chop it up. This means you can follow best practice and optimise the read-only cache. Disks in the data pool should be using read-only caching but the disk on the log pool should be set to none.

It is important to understand that like disk offerings VM offerings also come with various assocciated performance options that mean lower price and less performance or the opposite. This applies to what storage a VM can consume.

VM disk quanity limits

A VM SKU will have a maximum number of managed disks that can be allocated to it for example, a Standard_D2s_v3 can have a maximum of 4 managed disks.

VM disk throughput and IOPS limits

There is also a limit on storage performance for each VM type. So you could have the most optimized storage pool with fantastic IOPS and MBps throughput but if the VM using it cannot consume the performance it will be capped at the VM limit. For a Standard_D2s_v3 the maximum uncached IOPS is 3200 and throughput is 48MBps.

Create a storage pool

Add new managed disks to the VM

In the Azure portal add the new managed disks you want to use in the pool. The disk should all be the same size and SKU type. To ensure the disk as a resource is unique use the same naming standard as managed disks and append ‘-sp-disk#’.

Create the storage pool

1. On the server open Server Manager > File and Storage Services > Volumes > Disks
2. From Tasks select Rescan Storage
3. Now go to Server Manager > File and Storage Services > Volumes > Storage Pools
4. From Tasks select New Storage Pool
   • Give it a name such as “Data Storage Pool for drive L”
   • Select the Primordial pool of disks which should be the ones you just added in the portal
   • Select the physical disks you want in the pool -Leave the Allocation as default unless you want to change it -Review the options and select create

Create the virtual disk

1. In Server Manager > File and Storage Services > Volumes > Storage Pools
2. Right click the new storage pool and select New Virtual disk
3. Select the storage pool and click OK
4. Give the virtual disk a name such as “Data virtual disk for drive L”
5. Ignore enclosure awareness
6. In Storage layout select Simple
7. In provisioning select Fixed
8. In size select Maximum Size
9. Review the options and select create

Create a volume

1. In Server Manager > File and Storage Services > Volumes
2. From Tasks select New Volume
3. Select the new virtual disk
4. Make sure the size matches the virtual disk
5. Choose a drive letter
6. Change the allocation if required
7. Enter a disk label such as “Data”
8. Review the options and select Create

Add disks (expand) an existing storage pool

First add the new managed disk in the Azure portal then on the VM rescan the storage which should bring up the new managed disk. Right click the storage pool and choose to add a physical disk which will just add the new disk to the pool making the pool larger.

Next, open PowerShell as Administrator and run the following command which will show you the storage pool size and allocated size; the size value will still be the same.

Get-StoragePool

In the same PowerShell session run the following commands to tell Windows to take advantage of the new disk you’ve added to the pool.

Get-StoragePool [storage_pool_name] | Optimize-StoragePool

This will take a while to complete so you can monitor progress by using the command

Get-StorageJob

When the optimization is complete then run the Get-StoragePool command again to check the size value has increased.

Extend the virtual disk

1. In Server Manager > File and Storage Services > Volumes > Storage Pools
2. Right click the virtual disk and select Extend Virtual Disk.

Extend the volume

1. In Server Manager > File and Storage Services > Volumes > Disks
2. Right click the virtual disk and select Extend Volume.

References

https://docs.microsoft.com/en-us/windows-server/storage/storage-spaces/add-nodes https://www.itprotoday.com/iaaspaas/azure-disk-configuration-best-practices