Minimizing I/O when migrating from Virtualbox on Veracrypt to Proxmox on ZFS

I’m moving large-ish (half-TB or larger) files between hosts. It’s important to avoid extra copies in this workflow, since each pass over one of the larger files to read the whole thing (and there are several of them) takes hours. I managed to decrypt VM disk images, transform them from one disk image format to another, copy them from one host to another, calculate SHA-256 hashes on both sides to verify data integrity, compress and encrypt them on the destination, and to display a progress bar, all without any additional copies. One big block-device read on the source end and one big block-device write on the destination end is all of the disk I/O that’s happening.

See below for how I did this.

Continue reading “Minimizing I/O when migrating from Virtualbox on Veracrypt to Proxmox on ZFS”

Save power and heat: spin down backup drives when idle

Here’s a tip for those of you who, like me, back up your data to hard disks instead of tapes. Backing up to the same hard disk doesn’t protect you much (if the disk failed, you’d lose the data and the backup at once), so presumably you’re backing up to a separate physical drive. That means that the backup drive need not spin 24/7. Instead, it only needs to spin at backup time.
Continue reading “Save power and heat: spin down backup drives when idle”

Ubuntu Linux 7.10 “Gutsy Gibbon” Upgrade Report

A few weeks ago I updated to the latest version of Ubuntu Linux. This is the 7.10 (meaning October 2007) release, called “Gutsy Gibbon”. I encountered a couple of serious issues early on, but now that these are resolved things are running well. I’ll describe the issues and solutions so that anyone else encountering them can easily overcome them.
Continue reading “Ubuntu Linux 7.10 “Gutsy Gibbon” Upgrade Report”

Looking forward to LinuxWorld Expo SF 2007

Last year I was doing mobile development and there was interesting Linux-as-smartphone-OS stuff going on. Now I’m doing Ruby on Rails development and there’s interesting server grid stuff going on. Here’s what I’m looking forward to finding out more about (all of these are things I’ve been watching or directly researching already):
Continue reading “Looking forward to LinuxWorld Expo SF 2007”