As part of my role wearing the DevOps hat at DigiResults I’m in charge of setting up new machines; since I’m terminally lazy I clearly want to automate this. My first attempt used the ubuntu-vm-builder (which is a ubuntu-provided wrapper around the more generic vmbuilder script and was good enough for the first version but not without its drawbacks - namely I got bitten by making the VM disk too small.

vmbuilder will happily write to a ‘raw’ partition, which in my case was a LVM logical volume. The easiest way to show what I mean is by utilising my stunning ASCII-art skills. Running (not the complete command):

ubuntu-vm-builder kvm --raw /dev/mapper/vms-my_vm --hostname=my_vm

produces as disk layout the looks something like this (the PV and VGs aren’t shown as they just complicate matters

--- LVM on Host -------------------
| /dev/mapper/vms/my_vm (LV)      |
|                                 |
|    ----- VM ------------------- |
|    | /dev/sda*                | |
|    |                          | |
|    |                          | |
|    |                          | |
|    |                          | |
|    ---------------------------- |
-----------------------------------

The ‘hard disk’ that the VM sees is actually an LVM partition with an MBR and partition table etc. on it. Since /dev/sda is on LVM we can resize this ‘disk’ but since there is a partition table this is just as if you did a byte-for-byte copy (using dd or similar) of the old disk onto a new one - it doesn’t actually resize the partitions. Bugger.

I know I could have simply taken the VM down, run fdisk+ext2resize and then booted it up to work around this but I really wanted to do this without down time (and I couldn’t get round the kernel’s caching of partition tables when attempting the same on-line.)

So what option is there to have on-line-resizeable VMs? Run LVM in the guest and on the host. Yes, that will incur an extra (hopefully slight) performance hit, but since we’re talking about VMs here it shouldn’t be significant. As for it being a crazy idea: it certainly is, and it’s all Dean’s.

To round this post off here’s another limited edition Ash-ASCII-art image:

--- LVM on Host -------------------
| /dev/mapper/vms/my_vm (LV)      |
|                                 |
|    ----- VM ------------------- |
|    | /dev/sda*                | |
|    |                          | |
|    |   -- LVM on VM---------- | |
|    |   | /dev/mapper/lvm/*  | | |
|    |   |                    | | |
|    |   |                    | | |
|    |   ---------------------- | |
|    ---------------------------- |
-----------------------------------

Unfortunately I couldn’t work out how to get vmbuilder to put LVM onto the raw partition so I switched to netboot and preseeding which does support LVM (rather trivially) and is still fully automatic. Actually finding out all the pressed questions/answers to get it fully automated with LVM is another post.