Skip to content

Lions and Tigers and Unscheduled Outages, Oh My!

UNIX

 For those of you wondering where bsd-box.net went this weekend, here's the scoop:

A while back I stopped being a data protection douche and implemented backups.  I didn't blog about the implementation, but it was bacula writing to rsync.net over sshfs (fuse for the win).

These backups worked fine for a good while, but on Friday (my first full backup since upgrading to 8.0) the damn thing blew up and took the server down with it.  Near as I can tell without being on the console something caused the FUSE kernel module to go insane - The system goes unresponsive, starts taking forever to respond to pings and eventually falls over entirely.

Troubleshooting that mess is on my list as soon as I can reproduce the problem in a test lab, but until then bsd-box.net is running a ghetto-ass tar-then-SCP backup routine (so I'm not a data protection douche since I've still got backups, but my backups are decidedly more ghetto).

 

Those of you who sent me emails, panic not: I have a very fine secondary MX which will eventually despool whatever you sent me.  If it's urgent go ahead and resend.