A Tale of Two Servers

A few friends and relatives have expressed interest in learning why this site disappeared altogether earlier this month, then reappeared only to be replaced by a placeholder page for several days. Here’s the story as it unfolded.

On January 2, Saturday morning, we discovered early in the day that we couldn’t retrieve our email, nor was my web site online. This wasn’t an unprecedented situation–anybody who spends any amount of time online knows that there are the occasional outages–so I didn’t think much of it at first. After an hour or so I became disconcerted. I managed to find a status message at my domain host’s site that said the server which hosted my site had experienced a RAID failure, that restoration was in progress, it might take until 11:00AM on Monday for everything to be fully restored, but that I should check my site occasionally because they expected to have most accounts restored within a few hours.

By that evening we still had no email and no web site, so I started pestering their tech support people. It didn’t quite make sense to me that it would take so long to restore a server to active use. Eventually I got them to tell me that the situation was that they’d lost an entire RAID-5 because of a hardware failure and had to restore over 600GB of data from tapes to restore all of the user accounts on the server. They also gave me an estimate of how much progress had been made at that time. Given that estimate, it didn’t take any fancy math to figure out that there was no way that they were going to get everything running again by 11:00AM Monday.

So, we waited, and waited, and waited. No email, no blog. It was a slightly surreal experience for two people who are entirely accustomed to reading and sending email almost constantly throughout the day. (I suppose it would be like having your television taken away if you’re a regular TV watcher, but since we don’t have a television I can’t really compare the two situations with authority.) I did use a Gmail account to tell immediate family and some friends what was going on, but I deliberately didn’t try to email everyone that I could think of because I was laboring under the hope that my service would be restored Real Soon Now.

Altogether we were without email for about four days, and it took another day after that for my site to start working again because somehow they managed to botch the restoration of my files. All email sent to us during that time never reached us. Once I had email again I had to reassure various people like my financial advisor and the person who does Audio Damage’s payroll that yes, I was back among the online living.

This was all very inconvenient and rather vexing, but it would have been worse if the Paper Jade site had suffered the same fate. Thankfully that site was on a different server and hence unaffected. My ongoing conversations with the tech support department implied that there really wasn’t any reason to think that the same thing couldn’t happen to that site, even though it was a very unusual failure. Since this wasn’t the first time I’ve been displeased with this hosting service, I started a task I’ve been avoiding for at least a year: finding a new home for both domains.

This is a fairly daunting undertaking. There are many, many choices and it’s difficult to have any idea of the quality of the service until you’ve lived with it for awhile. Fortunately I know several people who have much more experience with such things than I do, so I solicited their opinions, shopped around a bit, and settled on Media Temple. (Hey, if they’re good enough for Sony, Toyota, and Warner Bros, they’re good enough for me.)

My cousin Ziggy accurately compared moving a web site to moving your home. You have to pack everything up, get it from the old house to the new house, unpack everything, and put it all in its proper place. Since I was going from a turnkey web-hosting service to a dedicated server the transition was even more complex. To follow the moving analogy, not only did I have to move everything, I also had to change the locks, replace the windows, knock out a wall while remodeling the living room, and learn how to be a carpenter in the process. Bear in mind that this was happening at the same time I was finishing up Tattoo and subsequently *cough cough* addressing a couple of issues that sprang up immediately after its release. Basically for a couple of weeks I was spending all day on Tattoo and all evening on setting up the server and moving the web sites, taking breaks only for necessities like eating, sleeping, and playing Call of Duty on my Xbox360.

I had planned to make a seamless, behind-the-scenes transition with both sites by getting everything set up on the new server before switching over (i.e. changing the DNS name server entries). The Paper Jade site made the transition quite gracefully: once I finally had everything in place and debugged (with some help from a friend) the change-over happened late in the afternoon last Saturday without a hitch.

It didn’t exactly work out that way for this site. Some afternoon partway through the whole ordeal–around the 19th, I think–I was immersed in working on the Paper Jade site and we again lost our studionebula.com email access and this site vanished from the web. I’ll admit that I panicked slightly and decided to switch studionebula.com over to the new server without further ado. I somewhat hastily set up the email accounts, put a temporary web page in place, and submitted the DNS name server change shortly before going to bed. The next morning everything seemed okay at first, but I soon figured out that I’d screwed something up with the server configuration and all of the email sent to us between about midnight and 9:00AM had been lost. Whoops. Fortunately the mistake was fairly easy to correct. But, since paperjade.com is the site that generates money and this one isn’t, I didn’t start working on getting this blog back up and running on the new server until I had the Paper Jade site running. That’s why if you visited this site between the 20th and the 25th you found only the temporary page.

So, paperjade.com and studionebula.com now live on a nifty server which I believe to be far more reliable than their old homes and much less vulnerable to long-term disaster. This site is still missing the older photo gallery which I abandoned in favor of Flickr. I intend to restore that next but I’m not making any promises or guesses about how long it will take me to get around to it. Setting up the server and moving the sites was all very interesting but frankly the novelty has worn off, and there is still some behind-the-scenes details I’d like to tidy up first, e.g. setting up a system for automated offsite backups.

By adam

Go ahead, try to summarize yourself in a sentence or two.

Leave a comment

Your email address will not be published. Required fields are marked *