Moving Servers At Dreamhost

After making this post, I submitted a support ticket to Dreamhost. Justin at Dreamhost replied back with a pretty solid sounding solution:

I did notice a few busy sites on the apaches for 2 of your domains. I started a move to another IP/apache which should complete in a few hours, and we can see if that apache turns out to be more stable. Since the server-status page for the old 2 showed an uptime of about 1 hour, likely the sites were causing that apache instance to time out and need to be restarted, resulting in a bit of downtime. Give it a try for a day and if you still have problems, let me know!

Perfect. Switch my domains over to a new httpd process with less activity. Makes sense. Well, after that was done, not much of an improvement was seen, as you can probably tell.

Today, I received another email from Justin saying he noticed longren.org was still slow to respond. He offered to move me to another server, assuming I was willing to have some downtime. He even asked if there was a time I’d prefer the move to be made. I asked him if 10:00 PST would work, and it did. So, this site is going to be offline for a few hours tonight while it’s being moved to a new server. The actual switch won’t take very long, I’ll just have to wait a few hours for the DNS to start pointing to the new IP.

Hopefully everything will be up and running along nice and smoothly tomorrow morning. Also, Justin at Dreamhost is awesome. He’s been very attentive and always offers possible solutions. Thanks again Justin!

Advertisements

Horrible Repsonse Time

longrenOrgEvenSlowerLongren.org has been really, really slow the last week or so. This site has never been this slow to load, even when I was hosting it out of my house on my cable connection. Granted, I didn’t get the traffic back then that I do now, it still shouldn’t be this slow.

Last time longren.org was being slow as shit, I posted an image of a graph from Site24x7, like I’ve done in this post. The response time in the earlier image is horrible, 4510 ms, but that’s a lot lower than I’m seeing now. As you can see from the image above in this post, the current average response time over the last 7 days is 6614 ms.

This has to be a result of something going on at Dreamhost. I say that because sometimes pages on longren.org will load up in a snap. Most of the time though they take between 15 and 30 seconds to load. Even sending queries to the database is slower than normal. Database queries are usually done being executed within 1 or 2 seconds. Lately, it’s been taking 5 to 9 seconds. Something is definitely up. Perhaps I will submit a support ticket to Dreamhost tomorrow. Yay.

More Dreamhost Network Issues

Dreamhost is recovering from yet another network outage. Apparently a switch had issues and they had to take a few servers off that switch. Supposedly the servers were moved over to another temporary switch so they’d have network access, but I haven’t had access to longren.org for at least 2 hours now.

We have moved the servers in the affected rack to knew switches in other racks temporarily while we get a new switch in place. We are working on recovery efforts now on servers which may be down. The list of affected servers is:

Arrow, mel, caesar, herod, alondra, overland, rossmore, oxnards, cerritos, selma, nala, demeter, jarvis. This is a mix of 3 MySQL servers and the rest webservers.

Longren.org is hosted on oxnard unfortunately, that box seems to have a lot of issues, even when the Dreamhost network is functioning properly. They’ve now managed to replace the problematic switch and are working on bringing servers back online.

Sorry about no timestamp before, it’s 12:36 Pacific time (-0800? I never could keep track of daylight savings time.)

We have completely replaced the switch and are working on getting the servers back online. All networking cables are back in their regularly scheduled switch ports.


I always get a kick out of the angry comments that show up on the Dreamhost Status blog whenever a fairly widespread problem occurs. Of course, Dreamhost hasn’t had great reliability in the last few months. I feel bad for those trying to run a business with Dreamhost.

One quick note, WordPress 2.0.5 should be here soon.

UPDATE: Apparently there’s still trouble. Now Dreamhost is saying the servers affected are still behaving abnormally. This box, oxnard, doesn’t appear to be affected though, for once! They’re still updating that same post at the Dreamhost Status blog:

Wierdness is afoot. The servers are still exhibiting the same problems, we have even moved two of them into our new datacenter and it’s the same deal. It’s only specific servers in half of one of our racks.

UPDATE 2 @ 8:52 PM CST: Dreamhost now says all their problems are over. Which they do appear to be. I believe the K2 site is hosted at Dreamhost, I noticed it was down for a while but has since been revived. Here’s the update from the Dreamhost Status blog:

As of 1700 Pacific time (GMT-0800 or so, 5PM), we think the issues with this rack are behind us. We moved some machines to new hardware and this seems to have fixed the problems. We will be looking in to this more on monday, and doing some extensive testing on that hardware to make sure it was the root cause of the problem.


Hopefully this was just a minor fluke and we don’t see any more problems. Like I said before, Dreamhost has been less than stable these last couple months. Honestly though, I have noticed this site loads much quicker since Dreamhost fought with their network last time. Still, there’s a lot of recent posts categorized as “system outages” at the Dreamhost Status blog.

Poor Site Performance

Poor Response TimeTake a look at the photo shown there. I don’t like it, at all. I’ve been using Site24x7 for the last few days to monitor various websites I run. It’s really handy and has brought to my attention a few things I wasn’t expecting. Namely, this site performs very poorly, which I’ve noticed more and more in the last few weeks.

OK, this site, longren.org, is hosted at Dreamhost. The Slackware Blog is of course, hosted at wordpress.com. And the other site in the image, is one for work. The site for work has better “response” times than both longren.org and the Slackware Blog. This took me by surprise because the work site is hosted on a Qwest DSL 1Mbit line.

Poor Response Time ComparisonI have a feeling the poor response time is mostly due to grabbing data from external sources, such as Google and Blogrolling. However, I don’t really include much content from external sources. Perhaps the small number of images causes this site to have much slower response times. The Slackware Blog and the work site are very light on images compared to this site. I never post images in posts at the Slackware Blog.

Anyway, I guess a 4500 millisecond response time isn’t too terrible, it’s about 4.5 seconds. Over the last couple days, the response time has gone down to around 2500 milliseconds, or 2.5 seconds. This site does have a larger amount of data to pull for every page, so I guess increased response times here are to be expected. Maybe I’m just worrying about nothing.

Shiny New Firefox Theme

So, I just installed the “nightly” Mozilla Firefox 2.0 Beta 2 RC1. It’s a whole lot more stable than Firefox 2.0 Beta 1 was.
tlongren
I really like this new default theme. It’s much prettier, although I don’t really like some of the icon choices.

In addition to the new theme, there’s also some apparently new functionality when downloading .exe files. Every time I download a .exe it pops up a window saying “You have chosen to open blahblah.exe”. Only, it won’t let you open it directly, you gotta save it to your hard drive somewhere first.

I think I am going to stick with this release until FireFox 2.0 Beta 2 officially comes out, because as I said before, this is just a nightly build with the Beta 2 name RC1 name. This release has yet to crash on me, where Firefox 2.0 Beta 1 crashed roughly 10 times each day. Looks like the Mozilla team has been busy.