ashberlin.co.uk - blog

Nesting LVM for resizeable VMs

2011-02-06T14:24:19+00:00

As part of my role wearing the DevOps hat at DigiResults I’m in charge of setting up new machines; since I’m terminally lazy I clearly want to automate this. My first attempt used the ubuntu-vm-builder (which is a ubuntu-provided wrapper around the more generic vmbuilder script and was good enough for the first version but not without its drawbacks - namely I got bitten by making the VM disk too small.

vmbuilder will happily write to a ‘raw’ partition, which in my case was a LVM logical volume. The easiest way to show what I mean is by utilising my stunning ASCII-art skills. Running (not the complete command):

ubuntu-vm-builder kvm --raw /dev/mapper/vms-my_vm --hostname=my_vm

produces as disk layout the looks something like this (the PV and VGs aren’t shown as they just complicate matters

--- LVM on Host -------------------
| /dev/mapper/vms/my_vm (LV)      |
|                                 |
|    ----- VM ------------------- |
|    | /dev/sda*                | |
|    |                          | |
|    |                          | |
|    |                          | |
|    |                          | |
|    ---------------------------- |
-----------------------------------

The ‘hard disk’ that the VM sees is actually an LVM partition with an MBR and partition table etc. on it. Since /dev/sda is on LVM we can resize this ‘disk’ but since there is a partition table this is just as if you did a byte-for-byte copy (using dd or similar) of the old disk onto a new one - it doesn’t actually resize the partitions. Bugger.

I know I could have simply taken the VM down, run fdisk+ext2resize and then booted it up to work around this but I really wanted to do this without down time (and I couldn’t get round the kernel’s caching of partition tables when attempting the same on-line.)

So what option is there to have on-line-resizeable VMs? Run LVM in the guest and on the host. Yes, that will incur an extra (hopefully slight) performance hit, but since we’re talking about VMs here it shouldn’t be significant. As for it being a crazy idea: it certainly is, and it’s all Dean’s.

To round this post off here’s another limited edition Ash-ASCII-art image:

--- LVM on Host -------------------
| /dev/mapper/vms/my_vm (LV)      |
|                                 |
|    ----- VM ------------------- |
|    | /dev/sda*                | |
|    |                          | |
|    |   -- LVM on VM---------- | |
|    |   | /dev/mapper/lvm/*  | | |
|    |   |                    | | |
|    |   |                    | | |
|    |   ---------------------- | |
|    ---------------------------- |
-----------------------------------

Unfortunately I couldn’t work out how to get vmbuilder to put LVM onto the raw partition so I switched to netboot and preseeding which does support LVM (rather trivially) and is still fully automatic. Actually finding out all the pressed questions/answers to get it fully automated with LVM is another post.

Colorful Ubuntu init.d scripts

2010-08-24T13:19:29+01:00

Writing init.d scripts is fun. So much fun that its really easy to fall down rabbit holes >_>.

Looking in /lib/lsb/init-functions at the various log functions available I noticed that the status reports should be in colour - after all I’m using a terminal that is reporting as xterm-color. It turns out that the log_use_fancy_output function doesn’t care about that variable (so long as TERM isn’t reported as “dumb”) but instead looks at tput and various capabilities from the terminfo database, and more specifically hpa which sets the horizontal position to an absolute value.

And horror of horrors its missing from the standard terminfo database. If this prints something for you then you can stop reading now - HPA is working on your terminal.

ash@home:~$ tput hpa 20 && echo a

Luckily its easy enough to add by running this command:

(infocmp; printf '\thpa=\\E[%sG,\n' %i%p1%d) > $$ tmp-${$}.tic && \
  tic -s tmp-$$.tic && rm tmp-$$.tic

What this does is dumps the current terminfo, adds the capability for moving the cursor horizontally to an absolute position, and then compiles the entry descriptions to a format usable by ncurses and tput. (I used printf since the echo binary on OSX doesn’t support the -e flag to escape input.)

(If you want to do this system wide, and a -o/etc/terminfo option to the tic command.)

Now lets test it:

ash@home:~$ echo -n b && tput hpa 20 && echo a
b                   a

With any luck the init.d scripts on Ubuntu will now print [ OK ] in colour on the right hand side of your terminal.

Host header with nginx proxy_pass

2010-03-31T12:37:17+01:00

I recently moved this blog and evilstreak’s to a VM on my stonking new server from Hetzner: 24GB. Woo!

But I made a booboo in switching from lighttpd to nginx. It turns out that the proxy_pass directive for nginx overwrites the Host header from the request with the proxy host. This in turn means that jsgi-vhost will fallback to the default host: in this case it happened to be evilstreak’s blog. Oops. The correct config invocation for nginx to preserve the host header from the request is:

server {
    listen 80;
    server_name ashberlin.co.uk;

    location /blog/ {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $http_host;
    }
}

So to those of you who read my RSS feed - sorry for the confusing posts over night. That’s what I get for playing at being a sysadmin late at night.

Running JSGI Multiple Apps in One Process

2010-02-10T00:00:00+00:00

Good idea or bad?

‘App’ in this case is talking about running two different Pressed (a blog engine for Juice) apps. More generally this question could apply to any application written in a scripting language.

Well on the one hand, if the apps use a lot of common modules, the memory footprint will be reduced. And using less memory is always hand for resource-starved virtual servers.

However more seriously this turns out to be a horrendously bad idea. Who knew? (Clearly not me). Even when you control all the apps in question and can set aside the security concerns, there are other reasons to not do this. Well just one really: its far to easy to unintentionally share state between the apps apps, even when you know about this danger and thought you’ve worked around it.

This is just a post of the “Remember Ash, Don’t Do That” variety. Some approach like this node.js patch might be a suitable alternative, and it gives other fuzzy warm benefits too.

Markdown's List Handling

2010-01-23T20:53:15+00:00

Jeff Atwood of codinghorror.com recently posted a rant/diatribe against Markdown, whilst at the same time declaring his love for it…

I agree 99% - the 1% where I don’t agree is point #3 is a non-issue for me.

When evilstreak and I wrote markdown-js recently we spent hours ranting against the fact that the ‘specification’ for Markdown is the broken implementation of Markdown.pl. (As for why we wrote a new implementation, well thats a whole other story. Main reason being needed an intermediate representation to play with.) Take this particular gem:

  * foo
   * bar
* baz
 * fnord

Its rather brilliantly ambiguous. It is of course not mentioned in the ‘specification’ (which is not a specification per say: its a usage guide). Markdown.pl treats it as if you’d written:

* foo
  * bar
  * baz
  * fnord

So of course this is the behaviour in most markdown implementations – Babelmark is a good tool for this sort of checking. If you think that case is odd, try to work out what this example is treated as:

   * foo
  * bar
 * baz
* HATE
  * flibble
   * quxx
    * nest?
        * where
      * am
     * i?

Any guesses? No? Well, its as if you entered this:

* foo
    * bar
    * baz
    * HATE
    * flibble
* quxx
    * nest?
        * where
        * am
        * i?

If you dont believe me, try it for yourself. This took me a number of hours just staring at the output and trying different permutations to to work out just what was going on here. I guess I could have looked at the code, but I like my eyes, thank you very much.

I’m not sure if this is how other implementations manage it, but what we did to replicate this (brain dead) behaviour was:

keep a ‘stack’ of previous lists and what indent they have, updated when ever you create a new nested list or ‘un-nest’ a list item by pushing and popping
if the next item is indented by 4 more spaces than the ‘indent level’ of the previous item then it’s a nested list.
if the number of indent chars (after tab expansion) exactly matches the indent of a list in the stack, place it at the same level as that item
failing that, treat it as if it was just indented to make it a sub list, even if it is actually indented less than the ‘parent’ list item.

The third point above explains why in case VII quxx appears at the same level as foo. And screams out for a desperate need for some cases to be specified as errors.

So no you know what you get when you have people implement a project in different languages without a sufficiently spec-like-spec or even some regression tests they could run: chaos.