Debian guest on Windows host using discard/TRIM.
This isn't a direct answer per se, as I'm addressing the problem, not the question. Instead of periodically compacting the image, this solution uses discard to automatically remove unused blocks in the host's VM disk image.
This solution requires a guest filesystem that supports continuous TRIM.
The Arch Linux wiki has a list of filesystems supporting TRIM operations.
FDE and cryptoroot are specifically not covered, as there are security concerns and none of the other solutions to this question would allow compacting either. The Arch Linux wiki has information about TRIM and dm-crypt devices.
In theory, this will work for all Linux guests on VBox hosts using VDI storage.
Host configuration
With VBox exited and no VMs running, add discard support to your disks by setting both discard and nonrotational for each disk in the config file for the VM. At this time discard is not in the GUI, but nonrotational is exposed as the "Solid-state Drive" checkbox. (ref: vbox forums, discard support)
<AttachedDevice discard="true" nonrotational="true" type="HardDisk" [..other options..] >
Boot the VM up, and verify that TRIM support is enabled:
sudo hdparm -I /dev/sda | grep TRIM
Guest Configuration
If LVM is in use, change the discard setting in /etc/lvm/lvm.conf. (ref: debian wiki, lvm.conf example)
devices {
...
issue_discards = 1
}
In fstab, add the discard option to the filesystems you wish to auto-discard (ref: debian wiki, fstab example)
UUID=8db6787f-1e82-42d8-b39f-8b7491a0523c / ext4 discard,errors=remount-ro 0 1
UUID=70bfca92-8454-4777-9d87-a7face32b7e7 /build ext4 discard,errors=remount-ro,noatime 0 1
Remount the filesystems to have them pick up their new options.
sudo mount -o remount /
sudo mount -o remount /build
Manually trim free blocks now with fstrim. fstrim uses the mounted filesystem, not the block device backing it. Instead of setting continuous discard in fstab, this could be done on a weekly cron. (The weekly cron is recommended for physical SSDs which may have questionable support for TRIM, but this is not relevant here since underlying SSDs are handled by the host OS. see: ssd trim warning).
fstrim /
fstrim /build
At this point, the size of the filesystems inside the VM and the size of the VM images should be pretty close in value.
Tested with:
- Guest1: Debian 8.7, kernel: linux 4.8 grsec from backports, filesystem: ext4
- Guest2: Debian 9 RC2, kernel: linux 4.9, filesystem: ext4
- Host1: VBox 5.1.14, Win7, image fmt: VDI
- Host2: VBox 5.1.14, Win8.1, image fmt: VDI
ddwith no argument copies one block at a time, not one byte. Using a larger block size can speed things up, but the effect is pretty minor. – Gilles 'SO- stop being evil' Aug 27 '13 at 00:15Zerofree is available on Ubuntu Linux via apt, or you can compile it yourself.
– Dakatine Feb 20 '14 at 13:16/bigemptyfile? A special path recognized bydd? – Frozen Flame Apr 30 '14 at 06:24sudo dd if=/dev/zero of=/zero.fill bs=4096kis functionally exactly the same, it will just write to a file namedzero.fillinstead (you'd need to change thermcommand to remove this file instead of/bigemptyfilethough) – Jason Larke May 05 '14 at 05:01dd ...; rm /bigfile, this will minimize the time with a full disk in case you're not waiting for theddto complete. – jlh May 14 '14 at 17:45dd: error writing ‘/bigemptyfile’: No space left on device
– Vincent Nov 12 '14 at 13:16/ is busyonmountcommand, then reboot your guest in recovery mode. – Vasilly.Prokopyev Nov 30 '15 at 14:22ddoperation will take a while, you can see its progress installingpvand usingsudo dd if=/dev/zero | pv | sudo dd of=/bigemptyfile bs=4096k. Source: http://askubuntu.com/questions/215505/how-do-you-monitor-the-progress-of-dd – Katu May 18 '16 at 09:00sdeleterequires a target as argument. Examplesdelete.exe c: -z– Francesco Frassinelli Jul 08 '16 at 07:30VBoxManagealso on Linux, notvboxmanage. – Andrea Lazzarotto Aug 15 '16 at 21:08telinit 1didn't work for me in Ubuntu, so I rebooted holding down the left shift to get the grub menu, I booted in recovery mode and in the recovery menu I selected the root shell. After that themountand thezerofreecommands worked like a charm. VM reduced from 25 GB to the expected 11 GB. – f.ardelian Aug 19 '16 at 19:39sudowhen writing zeros to disk on a Linux guest. Just write to /tmp/bigemptyfile instead of /bigemptyfile:dd if=/dev/zero | pv | dd of=/tmp/bigemptyfile bs=4096k; rm /tmp/bigemptyfile– sbleon Sep 27 '17 at 17:00vboxmanageandVBoxManage) of the tool. Even more, they are just symlinks to a real executable: commandls -o $( which vboxmanage VBoxManage )produceslrwxrwxrwx 1 root 4 Sep 13 16:37 /usr/bin/VBoxManage -> VBoxandlrwxrwxrwx 1 root 4 Sep 13 16:37 /usr/bin/vboxmanage -> VBox. – Victor Yarema Oct 10 '17 at 10:19ddto write 85Gb?!? – Merc Nov 25 '17 at 05:47ddcommand suggested here; https://askubuntu.com/a/903178/516133 – Lee Jan 12 '18 at 12:23VBoxManage: error: Cannot register the hard disk '/path/to/thedisk.vdi' {eee104d7-cd65-402f-b816-a3be4ac30eb3} because a hard disk '/path/to/thedisk.vdi' with UUID {eee104d7-cd65-402f-b816-a3be4ac30eb3} already existsVBoxManage: error: Details: code NS_ERROR_INVALID_ARG (0x80070057), component VirtualBoxWrap, interface IVirtualBox, callee nsISupportsVBoxManage: error: Context: "OpenMedium(Bstr(pszFilenameOrUuid).raw(), enmDevType, enmAccessMode, fForceNewUuidOnOpen, pMedium.asOutParam())" at line 179 of file VBoxManageDisk.cpp– Ed Randall Jun 10 '19 at 11:59swapoff -ain addition to booting in recovery mode and disabling systemd services. – Dan M. Aug 29 '19 at 16:40VBoxManage.exe: error: Cannot register the hard disk 'thedisk.vdi' {aaaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeee} because a hard disk 'thedisk.vdi' with UUID {aaaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeee} already exists Simply run the command by the UUID instead of the filename:
VBoxManage.exe modifyhd {aaaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeeee} --compact
– DustWolf Feb 27 '20 at 10:09modifymediumon the VDI. – kontextify Mar 27 '23 at 05:59