Lions and Tigers and Unscheduled Outages, Oh My!
For those of you wondering where bsd-box.net went this weekend, here's the scoop:
A while back I stopped being a data protection douche and implemented backups. I didn't blog about the implementation, but it was bacula writing to rsync.net over sshfs (fuse for the win).
These backups worked fine for a good while, but on Friday (my first full backup since upgrading to 8.0) the damn thing blew up and took the server down with it. Near as I can tell without being on the console something caused the FUSE kernel module to go insane - The system goes unresponsive, starts taking forever to respond to pings and eventually falls over entirely.
Troubleshooting that mess is on my list as soon as I can reproduce the problem in a test lab, but until then bsd-box.net is running a ghetto-ass tar-then-SCP backup routine (so I'm not a data protection douche since I've still got backups, but my backups are decidedly more ghetto).
Those of you who sent me emails, panic not: I have a very fine secondary MX which will eventually despool whatever you sent me. If it's urgent go ahead and resend.