Sometimes All The Testing in the World Doesn’t Help
Over the last several weeks, we’ve written several posts in this space about the importance of being prepared when it comes to launching web sites or applications. That means stress testing and other sand box scenarios to try to the best of your ability to make sure that your web site or application can handle whatever the world throws at it.
And that’s all well and good as far as it goes, except that it’s impossible to plan for every contingency. That’s because it’s impossible to reproduce every scenario that could possbily happen in a test bed. It just can’t be done, no matter how thorough you might be.
There are just way too many variables that you can’t account for from equipment to the vagaries of Internet connections to traffic variables. As we reported recently in the post, USAgov.jobs Relaunch Hampered by Performance Issues, the government claimed that the problems were due to volumes five times above anything they had experienced on the previous version of the web site.
If you believe the excuse, it’s a situation that was well beyond what any stress tests would likely have caught. Why would you test for five times the volume of any previous traffic spike? You tend to test for things that are likely to happen and you don’t usually plan on something completely out of the ordinary happening.
Even if you have a kitchen sink testing style — throwing in everything — you can’t possibly come up with everything because the one thing you didn’t consider might be the one thing that causes your application or web site to break.
If you could, you wouldn’t need monitoring tools. You would have figured everything out before you launched, and would have a plan for absolutely everything that could wrong. Since that’s not really realistic, you monitor your applications and web sites and watch for that unexpected situation.
Monitoring lets you see exactly what’s going on right now on your system and that means if it’s overloading or slowing down, or in the worst case that it’s stopped working completely, you can use those tools to track down the nature of the problem.
IT can expected to be wizards when it comes to keeping your web site and applications running 24/7, but you can expected to take reasonable precautions in the testing phase and to watch those critical applications and web sites when they are live.
You absolutely should be fastidious about testing and monitoring, maybe even a little neurotic, but know that no matter how anal you may be about it, sometimes you just have to give yourself a break because in some instances, all the testing in the world isn’t going to pick up the on problem you didn’t test for.