June 18, 2011

Comcast "business class" internet FAIL, when static IPs aren't

I have Comcast business class internet service (with a static IP) at home, and I've been pretty happy with that service up till now. But yesterday, my wife (who works from home) texted me saying our internet connection was down. From work I discovered that my server (which is hosted at my home using a static IP) was no longer accessible. Also, I was unable to ping the next hop Comcast router normally used by my internet connection.

Since I couldn't ping the next hop gateway for my normal connection I figured this must be a connectivity problem with Comcast. I told my wife to call tech support and she spoke to some very less than helpful person (that's my description, hers was more colorful). They told her there were no problems with our internet connection. When my wife insisted that things weren't working, he replied, "I am not saying you're lying, but I've refreshed my screen three times and your internet is working fine on my end." He then said he'd be happy to "roll a truck" for a business account customer, but since the problem was obviously on our end, they would likely charge us for it. He also stated that they had many people in the queue waiting for help and maybe she should call back when she knew what the problem was. She hung up.

When I got home, I logged into our cable modem and discovered that it did indeed seem like we had an internet connection. The cable modem had negotiated an upstream connection and could ping other hosts on the internet. Digging further into it, I discovered that the modem was indicating that we were hosted on a different subnet block, with a different static IP address. As mentioned before, I have a static IP address (which I pay extra for) because I run a server at home. In general, when you run a server you want to have an static IP address so that you can configure services to use that address. (For those not familiar with IPs, the easiest way to think of them is as phone numbers.  You give them out your number and people know that they can use those numbers to contact you.) But in this case Comcast changed my IP (which I had been using for the past two years) without notifying me.

I called Comcast support, pointed out the problem, and the support folks spent about 30 minutes taking me on and off hold while they confirmed that:
  • Yes, my static IP block had been changed earlier in the day.
  • No, there was no attempt to notify me that the change had been made.
  • No, there weren't any notes on my account indicating that this change had been made. So, had I not diagnosed the problem myself, their tech support would have had no way to see that there was any problem at all. This kinda explains (but does not justify) my wife's wonderful support experience with them.
  • No, they couldn't restore my IPs because they had already been re-assigned to someone else.
The speculation from Comcast tech support folks was that my previous IPs had been taken back for consolidation into a larger subnet block. I can understand this technical justification. (Having been a network administrator I'm well aware that larger network blocks require contiguous addresses.) And had I been contacted in advance, I would have been more than happy to plan to update all my router settings, server configurations, DNS servers, etc. But instead, no notifications were made and I had to diagnose the situation myself. This really doesn't seem like "business class" service to me.