Growing the disk in a vagrant basebox #

2017-03-31

I love using vagrant when making big changes to my puppet code. It allows you to really easily start with a blank slate for each iteration. It gives me a reasonable amount of certainty that should^H^H^H when one my instances blow up, a single puppet pass will do everything needed to generate a new one.

So, today, I’m happily working away on a major restructure of some puppet code when I run into this error.

Error: Execution of '/usr/local/sbin/pkg install -qy javaservicewrapper' returned 3: pkg: Not enough space in /var/cache/pkg, needed 467 KiB available -100 KiB

Ah crap, I’ve run out of space in the vagrant instance. This basebox only has a 4GB root.

$ df -h
Filesystem        Size    Used   Avail Capacity  Mounted on
/dev/ada0p2       3.7G    3.4G   -1.5M   100%    /

I need to make a new basebox with more space. One option is to just create a brand new basebox from scratch. But, instead, I decide to try and resize the disk in this existing basebox.

First of all, I created a minimal Vagrantfile. I just want to spin up a new VM based on my existing basebox image as a starting point so I run vagrant up with this Vagrantfile.

Once the instance comes up, I stop it with the virtualbox cli. Note, the VM that the Vagrantfile created is called ‘basebox’.

$ VBoxManage controlvm basebox acpipowerbutton

Lets see how the disk look first.

$ VBoxManage showvminfo basebox | grep IDE
Storage Controller Name (0):            IDE
IDE (0, 0): /home/chrskly/VirtualBox VMs/basebox/box-disk1.vmdk (UUID: ab38bc49-9e84-4635-9035-188038496198)
IDE (1, 0): Empty

The disk I want to resize is a vmdk. The virtualbox cli can’t resize vmdk images so we have to convert it to a vdi, grow it, then convert it back to a vmdk.

$ cd ~/VirtualBox\ VMs/basebox/
$ VBoxManage clonehd box-disk1.vmdk box-disk1.vdi --format vdi
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone medium created in format 'vdi'. UUID: c2eb30af-96b3-4224-b26f-232d9e382589
$ VBoxManage modifyhd box-disk1.vdi --resize 32768
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
$ VBoxManage clonehd box-disk1.vdi box-disk1-32gb.vmdk --format vmdk
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone medium created in format 'vmdk'. UUID: f8760621-d8bb-441f-a394-7509e26e5986

Now we have a clone of our disk image, only it’s 32GB instead of 4GB. We need to attach this disk to our VM.

$ VBoxManage storageattach basebox --storagectl "IDE" --device 0 --port 0 --type hdd --medium /home/chrskly/VirtualBox\ VMs/basebox/box-disk1-32gb.vmdk
$ VBoxManage showvminfo basebox | grep IDE
Storage Controller Name (0):            IDE
IDE (0, 0): /home/chrskly/VirtualBox VMs/basebox/box-disk1-32gb.vmdk (UUID: f8760621-d8bb-441f-a394-7509e26e5986)
IDE (1, 0): Empty

Note that there is no detach command for vbox storage. To replace a disk, you attach the new disk to old device/port address. This kicks off the old disk.

Now I want to boot the instance into single user mode to resize the filesystem. Because I’m lazy and don’t want to walk into the next room, turn on a monitor and type a few commands there, I set up serial console access to the VM. First, I had a look at the serial port config for the VM.

$ VBoxManage showvminfo basebox | grep -i uart
UART 1:          disabled
UART 2:          disabled
UART 3:          disabled
UART 4:          disabled

The serial port is not enabled. We need to enable it.

$ VBoxManage modifyvm basebox --uart1 0x3F8 4
$ VBoxManage modifyvm basebox --uartmode1 server /tmp/basebox.raw

0x3F8 + IRQ4 is COM1. You can get more info on this in the virtualbox docs here: https://www.virtualbox.org/manual/ch08.html#vboxmanage-modifyvm-other https://www.virtualbox.org/manual/ch03.html#serialports

We then use socat to create a pty and connect to it with screen.

socat UNIX-CONNECT:/tmp/basebox.raw PTY,link=/tmp/basebox.pty &
screen /tmp/basebox.pty

When you boot your VM, you should now see console output in your screen session. You may need to set console="comconsole" in your /boot/loader.conf in your VM.

Press 2 for single user.

Now lets grow the disk from the OS-es perspective. We have a swap partition here that we need to delete temporarily while we grow the root partition.

# gpart show
=>     34  8388541  ada0  GPT  (32G) [CORRUPT]
       34     1024     1  freebsd-boot  (512K)
     1058  7965696     2  freebsd-ufs  (3.8G)
  7966754   419840     3  freebsd-swap  (205M)
  8386594     1981        - free -  (991K)

# gpart recover ada0
ada0 recovered
# gpart show
=>      34  67108797  ada0  GPT  (32G)
        34      1024     1  freebsd-boot  (512K)
      1058   7965696     2  freebsd-ufs  (3.8G)
   7966754    419840     3  freebsd-swap  (205M)
   8386594  58722237        - free -  (28G)

# gpart delete -i 3 ada0
ada0p3 deleted
# gpart show
=>      34  67108797  ada0  GPT  (32G)
        34      1024     1  freebsd-boot  (512K)
      1058   7965696     2  freebsd-ufs  (3.8G)
   7966754  59142077        - free -  (28G)

# gpart resize -i 2 -s 31g ada0
ada0p2 resized
# gpart add -t freebsd-swap ada0
ada0p3 added
# gpart show
=>      34  67108797  ada0  GPT  (32G)
        34      1024     1  freebsd-boot  (512K)
      1058  65011712     2  freebsd-ufs  (31G)
  65012770   2096061     3  freebsd-swap  (1.0G)

# growfs /dev/ada0p2
It's strongly recommended to make a backup before growing the file system.
OK to grow filesystem on /dev/ada0p2, mounted on /, from 3.8GB to 31GB? [Yes/No] yes
super-block backups (for fsck_ffs -b #) at:
 8975872, 10258112, 11540352, 12822592, 14104832, 15387072, 16669312, 17951552,
 19233792, 20516032, 21798272, 23080512, 24362752, 25644992, 26927232,
 28209472, 29491712, 30773952, 32056192, 33338432, 34620672, 35902912,
 37185152, 38467392, 39749632, 41031872, 42314112, 43596352, 44878592,
 46160832, 47443072, 48725312, 50007552, 51289792, 52572032, 53854272,
 55136512, 56418752, 57700992, 58983232, 60265472, 61547712, 62829952, 64112192

Time to reboot.

# reboot
Mar 31 22:59:27 init: single user shell terminated.
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 0 0 0 0 0 0 0 done
All buffers synced.
Uptime: 56m7s
$ vagrant ssh
Last login: Fri Mar 31 22:02:36 2017 from 10.0.2.2
FreeBSD 10.3-RELEASE-p11 (GENERIC) #0: Mon Oct 24 18:49:24 UTC 2016

$ df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ada0p2     30G    2.5G     25G     9%    /
devfs          1.0K    1.0K      0B   100%    /dev
fdescfs        1.0K    1.0K      0B   100%    /dev/fd

Excellent! We can now package our VM to make a new basebox and I can get back to my puppet code.

$ vagrant package basebox
==> basebox: Attempting graceful shutdown of VM...
==> basebox: Clearing any previously set forwarded ports...
==> basebox: Exporting VM...
==> basebox: Compressing package to: /home/chrskly/Code/basebox/package.box