The Importance of Disk Alignment

Submitted by jbredehoeft on Mon, 12/05/2011 - 19:15

Disk performance is one of the bottlenecks that can adverse affect the performance computer system. This is true if you are running a physical system a Virtual Machine on datastore. All traditional rotating disks are divided into physical sectors. The operating systems and their components operate according to this sector logic. Since virtualization relies heavily on SAN storage, the alignment of the Virtual Machine's disk with the SAN become important. This is known as disk alignment. Some metrics will state a 9-15% improvement in performance with aligned disks vs. non-aligned disks.

The problem exists because the initial disk offset which is typically 4K sectors and emulated as 512B blocks does not match the sector layout on the storage provided by your SAN manufacturer. Usually the first 63 x 512b sectors of the VM disk comprise a hidden section of the disk. This creates an alignment that is not on the 64K block border, and means that the OS data blocks will cross the barrier of two storage or sector blocks. This process continues throughout the disk. In a SAN environment, the smallest hardware unit used by a SAN storage array to build a LUN out of multiple physical disks is a called a chunk or a stripe. To optimize I/O, chunks are usually much larger than sectors. Thus a SCSI I/O request that intends to read a sector in reality reads one chunk. Every time a block of the OS is read or written by the operating system it will actually cause two reads/writes by the SAN. This is further complicated when writing or reading multiple blocks that are not contiguous. Many more reads/writes occur than what is truely necessary if the OS filesystem in the VM disk is not aligned with the SAN storage chunks. Windows 2008 was created in a Virtualization world and so it doesn't suffer from this situation usually (see the footnote). Windows 2003 and earlier all have the block size problem, and so should be disk aligned. Every filesystem layer on the SAN needs to be disk aligned. See the attached diagram.

SSDs are gaining popularity with SAN storage and also in higher performance systems, servers, desktops and laptops. Despite the fact that SSDs store the data in a completely different way, they are still being treated with this sector logic. The alignment of the SSD is required to assure that a logical sector starts exactly at the beginning of a physical page of the SSD. Without the alignment, the sector boundaries and the page boundaries will not match and sectors will span pages. That would require for a Windows write operation to clear two blocks in lieu of only one thus reducing the write speed by 50%. All of the current storage devices are going to benefit from disk alignment (this includes RAID Partitions also).

How to do Disk Alignment

The disk must to be aligned be off-line. The ideal time to do alignment is before you build the system. However, you are likely reading this because you already have systems built, and you need to fix the alignment of those systems. I will cover both scenarios, the process to setup the alignment before you build the system, and how to correct alignment of systems that are already built. Deploy a virtual machine from a template that has already been aligned this requires no additional work and that virtual machine will forever more benefit from the affects of a correctly aligned volume.
Make sure you have working backups of existing systems. The potential exists to lose data.

Check VMFS Volume Alignment
I mention this because if you have been upgrading the VMFS volumes from VMFS 2 you have need to make sure that the VMFS Volumes are aligned. Beginning with ESX 3.x and later VMFS 3 is aligned on a 64K boundary. You will want to make sure that this matches the LUN stripe size of your SAN provider.

  • Log into your ESX host. You will have to be come root to do this.
  • list the partions. fdisk -lu
    Disk /dev/disks/vmhba2:0:0:0: 1999.8 GB, 1999839952896 bytes
    255 heads, 63 sectors/track, 243133 cylinders, total 3905937408 sectors
    Units = sectors of 1 * 512 = 512 bytes

    Device Boot Start End Blocks Id System
    /dev/disks/vmhba2:0:0:1 128 18446744073320515964 18446744073515033727+ fb VMFS

    The key here is the "Start Block", you can see this is on the 2nd 64K boundary. 2* 64 = 128.

If you find your VMFS is not aligned, there is little you can do. Depending on how much work is involved, you might consider destroying the data store and recreating it through the Virtual Infrastructure Client. If you decide to recreate your VMFS, be aware you will lose any data that exists on the partition.