Minimizing I/O when migrating from Virtualbox on Veracrypt to Proxmox on ZFS

I’m moving large-ish (half-TB or larger) files between hosts. It’s important to avoid extra copies in this workflow, since each pass over one of the larger files to read the whole thing (and there are several of them) takes hours. I managed to decrypt VM disk images, transform them from one disk image format to another, copy them from one host to another, calculate SHA-256 hashes on both sides to verify data integrity, compress and encrypt them on the destination, and to display a progress bar, all without any additional copies. One big block-device read on the source end and one big block-device write on the destination end is all of the disk I/O that’s happening.

See below for how I did this.

Continue reading “Minimizing I/O when migrating from Virtualbox on Veracrypt to Proxmox on ZFS”

OTP codes from Mac not working due to Mac clock running fast

I’m a fan of Time-based One-time Password, aka TOTP, as a means of Two-factor authentication. I’m also a fan of the 1password password manager, which I have set up to sync between my various devices: a laptop Mac, an Android phone, and an iPad. A nice feature of 1password is that it will act as an OTP authenticator, if you store the TOTP secret (either copy-pasted as text, or scanned as a QR code that decodes to a URL containing the same text) alongside your password info. So if I’m lying in bed with only my iPad nearby and I need an OTP code to log into something, I don’t have to get up and grab my phone to get it from Google Authenticator. (I do also have the OTPs in Google Authenticator though.) More importantly, this means that none of my devices is a single point of failure; if my phone is stolen or dies, I can still get into stuff because I have the secrets synced to multiple devices, encrypted in transit and at rest by the password manager.

But, this all stops working when your Time-based OTP is generated on a Mac with a clock running over a minute fast!

Continue reading “OTP codes from Mac not working due to Mac clock running fast”

Why security is hard, and why it’s not going to get easier

Bruce Schneier summed it up well: the good guys have to secure all the doors and windows; the bad guys only have to find one. In a nutshell, that’s why security is hard. Real-world security has to deal with that problem all the time.
If you think of writing software as filling a feature space, sometimes you accidentally provide functionality that you didn’t intend to, while providing what you meant to. You meant to provide an exhaust port but you also created a vulnerability.
Continue reading “Why security is hard, and why it’s not going to get easier”

Proper Error Handling in Rails Controllers

Rails controllers can get out of hand if you’re not very careful. Skinny Controller Fat Model is a great start. But what about handling errors? Isn’t it enough to just let Rails catch your exception and show a 500 Server Error page?

No, it’s not. Falling back on 500 Server Error for everything outside of the “happy path” through your code is sloppy coding.
Continue reading “Proper Error Handling in Rails Controllers”