This was a monumental failure, not only because it involved one of the most important technology defense departments, but it didn’t take out just the web site. Oh no, according to an Information week article, it took down some crucial services with it including time and attendance, training and file sharing. Ouch.
This follows problems with the USAJobs.gov site we reported on at the end of last month. And Information Week reported that Congress was literally making a federal case out of it, by convening a committee to look into the site’s problems.
You think you’ve got issues at your company in your IT Ops department? How would you like to have some Congressmen who controls your funding breathing down your neck about your outages? Come to think of it, a Congressman and an executive have about the same amount of knowledge about what it takes to keep your company’s server room (or data center) running smoothly – none.
But that won’t stop them from coming at you hard when an outage that affects the entire company stretches from hours into days. Luckily for DISA, FierceGovernment IT reports the site was finally back up by Wednesday afternoon, but that a single point of failure could cause so much damage had to be a bit startling to the folks in the DISA data center — and they probably will try to take steps to make sure it doesn’t happen again.
This was probably one of the worst case scenarios IT Ops pros can ever expect to see (at least you hope), because not only did it take down the company web site, it took down some significant applications too, meaning the outside world could get in touch with them and employees were severely hamstringed from doing their work as well.
And I’m sure as those days an hours stretched out in front of them, the pressure grew as more time passed. But if this ever happens to you, you can at least point to the DISA outage and say if it can happen to them, it can happen to anyone.
That might not make your CEO feel that much better about the down time, but at least he or she will know that your company’s not alone — and even pros need time to track the problem and restore the systems when things go terribly wrong.