I can hit any bank I want, any time. They got to be at every bank, all the time.
~Johnny Depp as John Dillinger in Public Enemies
When it comes to monitoring, almost really doesn’t count. That’s why if you’re using a monitoring tool that does automated system polling, you may be missing out. What happens during those times when the system isn’t polling still counts, and can actually be masking real problems you’re not seeing.
And that’s the problem in a nutshell with polling tools. You have to be watching all the time or you miss something. Much like Johnny Depp’s character in Public Enemies quoted above, you have to be in every bank all the time to catch the problems.
And if your bank gets hit while you’re not watching, you could be screwed. Of course, if your application performance deteriorates enough, you’ll definitely hear about.
Your Help Desk phone lines or online queues will fill up in a hurry with angry and anxious employees wondering why the application they need to complete their work is suddenly pokey or worse, not working.
As authors Alistair Croll and Sean Power pointed out in Chapter 10 of their book, Complete Web Monitoring, lots of stuff can happen when it looks like everything is operating fine. It could be you’re simply measuring the wrong thing, or it’s an issue that you hadn’t anticipated, like users accessing a web-based application using a different browser than the one or two you tested.
And if you are dealing with an intermittent problem, your polling software may show everything is working fine and your users just happen to be hitting the site when it’s not working in a very unhappy and unfortunate coincidence.
As Croll and Power write:
In other words, there are plenty of ways your site can be working and still be broken. As one seasoned IT manager put it, “Everything could be blinking green in the data center with no critical events on the monitoring tools, but the user experience was terrible: broken, slow, and significantly impacting the business.” To find and fix problems that impact actual visitors, you need to watch those visitors as they interact with your website.
If your job is to monitor the system, and the software you’re using has missed an important event, it’s embarrassing for you. Users (and more importantly) the managers and executives in charge of the idle employees are probably not going to want to hear that it’s a monitoring software issue.
And that’s the bottom line for monitoring professionals. The fact is it’s a rare shop that has a single monitoring tool in place, so you probably have more than one type watching your critical applications and network hardware, but those monitoring tools should be able to watch every bank all the time. If you’re not doing that, what you’re not seeing can definitely hurt you.