VirtualBox/KVM: Reduce VM sizes

There are two utilities that can help discard unused blocks so that VMs can be shrunk.

zerofree finds unused blocks with non-zero content in ext2, ext3 and ext4 filesystems and fills them with zeros. The volume can’ be mounted which makes the process of running it a bit convoluted.

fstrim will discard unused blocks on a mounted filesystem. It is best and preferred when working with SSD drives and thinly provisioned storage. It will work with more filesystems, and it won’t hammer your SSD with unnecessary writes.

It is recommended to use fstrim and only use zerofree if unavoidable.

CentOS 7/8

fstrim

# fstrim -va

zerofree (ext2, ext3, ext4)

# yum install epel-release
# yum install zerofree

[Reboot]
Press e on GRUB menu
Go to line that starts with 'linux'
Add init=/bin/bash
Ctrl-X

[Find which disk to trim]
# df
# zerofree -v /dev/mapper/centos_centos7-root

[Shutdown machine]

zerofree (xfs)

# yum install epel-release
# yum install zerofree

[Reboot]
Press e on GRUB menu
Go to line that starts with 'linux'
Change ro to rw
Add init=/bin/bash
Ctrl-X

[Find the partition/filesystem to trim]
# df

[Fill the filesystem with zeros. This will work with any filesystem but it will write a lot of data on your drives.]
# dd if=/dev/zero of=/tmp/dd bs=$((1024*1024)); rm /tmp/dd
# sync
# exit

[Shutdown machine]

Debian 9/10

fstrim

[Debian 9]
# fstrim -va

[Debian 10]
# fstrim -vA

zerofree

# apt install zerofree

[Reboot]
Press e on GRUB menu
Go to line that starts with 'linux'
Add init=/bin/bash
Ctrl-X

[Find disk to trim]
# df
# zerofree -v /dev/sda1

[Shutdown machine]

Ubuntu 18.04/20.04

[Ubuntu 18.04]
# fstrim -va

[Ubuntu 20.04]
# fstrim -vA

Be aware that if you are using ZFS on Ubuntu (or any other distro) the above commands won’t work. In fact, it will generate a lot of extra writes on the filesystem.

Just ensure that ZFS is using compression, or avoid it in the guest system.

Reducing the image size

Virtualbox

[List all disks]
$ vboxmanage list hdds

[Just the paths]
$ vboxmanage list hdds | grep  'Location.*.vdi' | awk '{$1=""}1'

[Compress one image]
$ vboxmanage modifymedium disk --compact /home/user/Virtualbox/Kali-Linux-2021.1/Kali-Linux-2020.4-vbox-amd64-disk001.vdi

[List all images path]
$ vboxmanage list hdds | grep  'Location.*.vdi' | awk '{$1=""}1' | sed 's/^ /"/;s/$/"/'

I wish I knew the syntax to automatise compressing all the images with one line. I might revisit it in the future with a script.

KVM

# qemu-img convert -O qcow2 originalfile compressedfile

I have a script to do all of the files in one go:

#!/bin/sh

# All images
for file_name in `ls -1 *.cow2`

do
	echo
	echo ==================
	echo Image: $file_name
	echo -n Old `qemu-img info $file_name | grep 'disk\ size'` ; echo
	mv $file_name $file_name.tmp
	qemu-img convert -O qcow2 $file_name.tmp $file_name
	rm $file_name.tmp
	echo -n New `qemu-img info $file_name | grep 'disk\ size'` ; echo
	echo ==================
done



Ubuntu 20.4: Virtualbox not running after the last upgrade

When launching a VM in Virtualbox I got an error saying that it can’t be started because a required module isn’t loaded. It suggests to manually load it.

# modprobe vboxdrv

[This outputs an error message]

modprobe: FATAL: Module vboxdrv not found in directory /lib/modules/5.8.0-34-generic

Re-installing Virtualbox also fails because the virtualbox-dkms package can’t be configured

# apt install virtualbox virtualbox-dkms
[...]
Removing old virtualbox-6.1.10 DKMS files...

------------------------------
Deleting module version: 6.1.10
completely from the DKMS tree.
------------------------------
Done.
Loading new virtualbox-6.1.10 DKMS files...
Building for 5.8.0-34-generic 5.8.0-36-generic
Building initial module for 5.8.0-34-generic
ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/virtualbox-dkms.0.crash'
Error! Bad return status for module build on kernel: 5.8.0-34-generic (x86_64)
Consult /var/lib/dkms/virtualbox/6.1.10/build/make.log for more information.
[..]
E: Sub-process /usr/bin/dpkg returned an error code (1)

The last system update upgraded the kernel from 5.4 to 5.8, and there is something in the new kernel that breaks Virtualbox.

There are two solutions:

Installing Virtualbox from source or downgrading to the previous kernel.

I have chosen the latter as I expect this to be a temporary issue, and a fix to be released soon.

The process to revert is simple.

Reboot and in the GRUB screen select Advanced Options.

Ubuntu 20.04.1 LTS
*Advanced options for Ubuntu 20.04.1 LTS
History for Ubuntu 20.04.1 LTS
UEFI Firmware Settings

Select a trusted 5.4 version to boot from. Most likely the 3rd option in the list. Your exact version numbers might differ from mine.

Ubuntu 20.04.1 LTS, with Linux 5.8.0-34-generic
Ubuntu 20.04.1 LTS, with Linux 5.8.0-34-generic (recovery mode)
*Ubuntu 20.04.1 LTS, with Linux 5.4.0-59-generic
Ubuntu 20.04.1 LTS, with Linux 5.4.0-59-generic (recovery mode)
Ubuntu 20.04.1 LTS, with Linux 5.4.0-54-generic
Ubuntu 20.04.1 LTS, with Linux 5.4.0-54-generic (recovery mode)

After the boot, check that you are running 5.4.

$ uname -r
5.4.0-59-generic

See which versions of 5.8 you have installed in your system.

$ apt list --installed | grep  linux-image

Make a note of the 5.8 versions listed (or use grep again), and remove them manually.

# apt remove linux-image-unsigned-5.8.0-34-generic

Virtualbox should be working.

Linux 5.8 seems to have been removed for the time being, so if you run any updates you are safe.