The Complete List of End User Experience Monitoring Tools
I am attempting to put together a complete list of End User Experience Monitoring tools since I could not find one anywhere on the web. I need your help in order to complete this list – there are surely going to be tools that I miss – please leave me comments with tools you think I should add to the list.
UPDATE: Due to overwhelming demand, this list is now available for download in spreadsheet form.
What Qualifies as an End User Experience Monitoring Tool?
In order to count as an End User Experience Monitoring tool it must be able to track the response times that real users experience when visiting the site – not a robot which is synthetically pinging the site. Specifically I am referring to tools that would enable IT operations to ensure that the real end users of an application or website are experiencing good performance. As I have alluded to in a previous post “speed solves a lot of problems” – claiming that even if your usability is not perfect – if it runs fast – people are less likely to notice.
If you would like me to remove a tool from the list – please leave me a comment.
| Vendor | Product | Description (Taken From Vendor Site) |
|---|---|---|
 | Real User Monitoring | SharePath RUM (real user monitoring) Express provides a real-time view into the actual experience of your end-user, including availability, response times and service levels. This free, enterprise-class software helps isolate problems and pinpoint bottlenecks in your data center, network, or online application. |
 | End User Experience Monitoring | Reduce the time, effort, and cost involved with monitoring dynamic applications by providing both IT and the business with real-time insight into application performance and its impact on user behavior — across physical and cloud environments, the internet, corporate networks, content delivery networks (CDNs), and mobile devices. |
 | Customer Experience Manager | CA Wily CEM focuses on identifying and prioritizing problems that affect end-user service quality by analyzing individual transactions in real-time; providing customized real-time dashboard displays that give you the information needed to make quick, accurate decisions; enabling business-based prioritization of problem resolution; and displaying information needed by both IT and business owners to work together to meet Service Level Agreements and deliver an optimum enduser experience. |
 | Real User Monitor | HP Real User Monitor software provides visibility into each user’s experience, transactions, and behavior for critical applications—at every location. Capture live sessions, see where customers clicked, measure response times, and see pages that caused problems. |
 | ITCAM for Transactions | ITCAM for Transactions provides an agentless Web response monitoring component with the ability to dynamically switch monitoring levels. The Web response monitoring component allows you to adopt an end user’s perspective when measuring transaction performance. End-user response time is filtered down to specific transactions by time, user ID, or session. |
 | Real User Experience Insight | Oracle Real User Experience Insight enables enterprises to maximize the value of their business-critical applications by delivering insight into real end-user experiences. It can help identify lost revenue from frustrated users, reduce support costs by lowering call center volumes, accelerate problem resolution of poorly performing applications, and help businesses adapt to changing needs by providing insight into business trends and user preferences. |
 | Customer Experience Management | Tealeaf's solutions are used by customer-centric organizations to build a true online customer experience competency across the enterprise. From ebusiness and IT, to customer service and compliance, Tealeaf provides the common "view" of the online customer enabling organizations to understand how to deliver better online solutions and provide more effective service for their customers. |
 | TrueSight End User Monitor | Acquired by BMC - but is TrueSight what BMC is referring to on their site as "End User Experience Monitor"? With TrueSight End User Monitor, organizations have the ability to view their Web applications and identify, through precise metrics, the health and viability of those applications as experienced by the actual end user. |
 | Foglight User Experience Management | Foglight® User Experience Management solutions help you understand and improve the experience users and customers have when interacting with your web applications. Proactively managing the user experience will help you identify performance issues, understand user and customer behavior, and improve web application performance. |
 | mod_pagespeed | This is not a fully featured enterprise monitoring tool - but it can collect the data. mod_pagespeed is a lightweight open source tool developed by Google. The 'Add Instrumentation' filter injects two small blocks of JavaScript into every HTML page. These blocks measure the time the client spends loading and rendering the page, and report that measurement back to the server. |
 | boomerang | This is not a fully featured enterprise monitoring tool - but it can collect the data. boomerang is a piece of javascript that you add to your web pages, where it measures the performance of your website from your end user's point of view. It has the ability to send this data back to your server for further analysis. With boomerang, you find out exactly how fast your users think your site is. |
 | Episodes | From the guy who wrote the book. This is not a fully featured enterprise monitoring tool - but it can collect the data. Episodes is a framework for timing web pages. It has three key concepts: it supports measuring Web 2.0 applications, measurements are made using JavaScript events so there can be multiple listeners, it has features that support making it an industrywide standard for web developers, web metrics service providers, tool developers, and browser developers. |
 | Jiffy-Web | Jiffy allows developers to; measure individual pieces of page rendering (script load, AJAX execution, page load, etc.) on every client, report those measurements and other metadata to a web server, aggregate web server logs into a database, generate reports |
 | Real User Monitoring | Now you can track real user experiences and app components in the same tool. We've hit the easy button for you. New Relic now has browser response time, slowest requests by browser or by geography, page-by-page performance breakdown, and more. Fully integrated, in one tool. |
 | Transaction Performance Management | Precise does not market their end user experience monitoring tool as a separate product, as far as I can tell from their website, but as part of the Precise "end-to-end" performance management platform. |
 | Gomez Real User Monitoring | Gomez Real-User Monitoring, also commonly referred to as passive monitoring, captures every dimension of application performance from the eyes of your end users, and provides crisp insight into the business impact of poor performance. |
 | AppResponse Xpert | AppResponse Xpert is an appliance-based solution that monitors and analyzes end-user experience for all levels of transactions. It also supports in-depth monitoring and analysis of the underlying network, a domain that is vital to comprehensive APM. |
 | Real User Experience Management | Frontline Performance Intelligence is the result of the real-time aggregation, analysis, correlation of all the performance metrics that define and impact real end user experience. By transforming end user experience metrics into actionable business intelligence, Frontline Performance Intelligence becomes a strategic business enabler. |
 | End-User Experience and Performance Management | Knoa's vision for EPM starts with monitoring application performance from the end-user perspective. Knoa's User Sensing Technology captures the end-user's interaction with an application at the GUI level rather than the transaction level, so it really measures true "end-user response time" and the actual "end-user experience." |
Alex Podelko just informed me that he had constructed a partial end user experience monitoring list – I will add his entries. If anyone else has a list – please let me know.
40 Comments
Trackbacks/Pingbacks
- Open Source Web Site Performance Tools | Real User Monitoring - [...] St-Amour wrote a very helpful comment on my “The Complete List of End-User Experience Monitoring Tools” post – helpful ...
- The 25 best new web performance links of Q3 — Web Performance Today - [...] The Complete List of End User Experience Monitoring Tools This is helpful list of tools for RUM and other ...
- Real User Experience | Softqanetwork.com - [...] http://www.real-user-monitoring.com/the-complete-list-of-end-user-experience-monitoring-tools/ [...]
Leave a Comment
Free Real User Monitoring
This blog is sponsored by Correlsense,
the maker of SharePath RUM Express -
a free tool that provides a real-time
view into the actual experience of
your end-user, including availability,
response times and service levels.
There’s quite a few others you could add… Here’s some off the top of my head:
CA – NetQoS
Compuware – Vantage & Gomez
Keynote Systems
OPNET Technologies ACE family
Aternity
OpTier
Webmetrics
Nimsoft
Tealeaf
Dynatrace
Netscout
Quest
Coradiant
Abbas – Thank you for all of the valuable additions to the list – quite a bit of work! I will make sure to update the list shortly – I have some questions:
Regarding NetQos – I already have Customer Experience Manager as Computer Associate’s end user monitoring tool – and when I go to the NetQos site and click on “CA NetQoS Super Agent End-to-end application response time monitoring” I get a page not found. Are you sure that NetQos still has an end user experience monitoring tool on the market?
Regarding Webmetrics – I am unable to identify which product does REAL end user experience monitoring – their “website monitoring” service does synthetics.
Regarding Keynote – do you know if their “Real Browser Monitoring” tool actually picks up on the end user experience of all users that arrive at a monitored site? Is that their end user experience monitoring tool?
For Netscout – are you referring to the “nGenius Performance Manager“?
Hi Alon,
This is a great list you have Created,I don’t have anything to add to list yet but really wanted to praise your work so that’s why leaving a comment you have really done a nice job it really helped me a lot in knowing how many tools are available for the purpose of RUM and i didn’t knew there were so many.
Hmm. I suppose other free options like OWA, Piwik (with plugins) and Google PageSpeed module, as well as Analytics’ SiteSpeed http://www.google.com/support/analyticshelp/bin/answer.py?answer=1205784 … aren’t really considered “enterprise” or “complete” … but I’m finding it harder and harder to get at a solution that “just works” in terms of stats/analytics, user experience pings/errors, as well as path tracking/analysis, mouse tracking and survey integration. I’m beginning to think that complicated setups like this require more than one tool with a lot of customization/configuration. Even the default usage of Google Analytics doesn’t go far enough in letting me inspect individuals’ paths in segmented ways, just aggregate data as a whole, by default. So I’m thinking starting with an open source product like OWA+PageSpeed and extending it to suit my needs might be best, when combined with data mining efforts.
Louis,
I am sure you are not referring to “Outlook Web Access” when you say OWA – please educate me.
Piwik looks like an analytics tool – can you refer me to where the end user experience monitoring feature is?
PageSpeed is a great free tool for improving the performance of your web pages – but does it measure the real end user experience of visitors to your site?
SiteSpeed looks cool – it is only a sampling – and I am sure that it keeps all user data anonymous – so it would be great for a simple web site – but not so useful when it comes to heavy duty enterprise applications as you say.
I agree that it is difficult to find a single tool out there that can do both the performance monitoring – down to the single user level – for any kind of application as well as all of the analytics that you would want to see.
Thanks for the reply, Alon. If it didn’t, it’d be nice if the comment system emailed me your reply, perhaps take a look at Disqus or other such comment systems, they really can improve community-building on blogs and news sites
OWA, indeed commonly means Outlook, so it’s kind of unfortunate naming clash, but in this case I’m referring to Open Web Analytics from http://www.openwebanalytics.com/ which has what it calls, “domstream recording”. This lets you see exactly what a user sees in their browser and where their mouse goes, though I’m not sure how embedded it is for performance timing or error catching. A similar feature is in piwik as a plugin, as I understand it.
PageSpeed is not just a Firefox/Chrome extension
It’s a web service now too! Seriously though, what I was referring to was mod_pagespeed, the Apache module, which beyond all its other fun goodies has an “instrumentation” feature that — you guessed it: measures “the time the client spends loading and rendering the page, and report that measurement back to the server” http://code.google.com/speed/page-speed/docs/filter-instrumentation-add.html
Google helpfully mentions that their instrumentation feature is not needed if you already have one of your own and cites Boomerang, Episodes or Jiffy:
- http://yahoo.github.com/boomerang/doc/
- http://stevesouders.com/episodes/paper.php
- http://code.google.com/p/jiffy-web/
Sampling is indeed the worst part about Google Analytics … sigh.
Also, I would suggest that “real end user experience” tracking is something you’d need to be Google Chrome to get. After all, as Google notes about its own filter: “Note that the data reported by this filter is only approximate and does not include time the client spends resolving your domain, opening a connection, and waiting for the first few bytes of HTML.”
Of course, that’s where the highly technical Speed Tracer comes in
http://code.google.com/webtoolkit/speedtracer/
Louis,
Do you mind if I use your comment as a blog post (-: it would be very helpful for our readers.
Thanks for the insight on OWA – I now do remember coming across it – the DOM stream recording feature is something I did not see and looks like a very helpful tool. To me this looks like more of an analytics tool that marketers and product designers would use in order to improve the usability of their product. I should probably clarify that I am referring to end user experience operations tools which allow for the scalable measurement of performance.
mod_pagespeed definitely looks like something I would put on the list as an open source alternative. I do remember hearing about it at Velocity. The other ones listed also look cool.
Thanks again!
ps – I am familiar with disqus – and I like it too – so I will look into installing it.
Sure, feel free to use whatever you like, just mention my name or link to my site (currently points at my G+ account).
To clarify, CC 3.0 BY – http://creativecommons.org/licenses/by/3.0/
New Relic recently launched their RUM (Real User Monitoring) with Real End User Metrics. We currently capture page load time for over 1.8 Billion pages every week, and capture 20 Billion unique web transactions everyday. The data is based on real end users – no synthetic data. The end user performance is presented in-line with core APM data for full visibility of the application’s experience. http://newrelic.com/rum
Thanks for the sales pitch Patrick (-:
I will add it to the list. BTW – I love that new infographic that you guys created.
Whoops – sorry for the salespitch version – It was a cut and paste.
Thanks – look forward to being on the list!
Ooh, New Relic, the Rails folks! Neat. I’m kind of confused how RUM relates to that of the pricing page … is there a free version, or is it a feature of one of the paid plans, or something else entirely?
We’re more than just Rails! We started in Rails but now support Java, .NET, PHP, Ruby & Python (Beta). RUM is included for free when you deploy our service in your web app. You have the option to auto-inject the async JS into your app’s header & footer. Its pretty sweet. (end sales pitch).
Hi – Good idea!
I think it might be good to break the list down somehow to highlight the different functionality available in each tool e.g robot/synthetic vs real user experience and also to highlight the enterprise tools that can map any sort of transaction across the enterprise.
There is a big difference between a point product that does robotic transactions for website performance management and a full end-to-end monitoring tool capable of pinpointing bottlenecks and problems inside complex applications and via thin client etc.
FireScope.com also have end user experience management as part of their BSM tool
Cheers
Mike
Mike,
Thanks for the input. I was thinking of doing a separate list for robotic transactions. It looks like this list is going to be rather long even though it is very focused.
Can you please send me the link to the FireScope End User Experience Management tool? I was not able to find it.
Regarding the “end-to-end” and transaction tracing tools – I think that may deserve s separate list as well (even though most of enterprise grade tools listed are part of a larger suite). Once I am done with listing everything out – I may add some commentary on each. I was thinking of even just simply segmenting them into software based vs. hardware appliance based tools.
i’m surprised knoa wasn’t on anyone’s list. I’m researching this space as well and would assume knoa (http://www.knoa.com) would be on it. (I’m not affiliated with them and have never used their software so don’t know how good they are, just been researching the different providers)
Dan – I am very familiar with Knoa – I even visited their beautiful offices once in New York’s Union Square. I will add them on – they probably have one of the most solid solutions for monitoring SAP applications.
Precise is another end-to-end monitoring tool that intorduced TPM for virtual appliance.
Chetana – Thank you – I guess TPM is the new name of the i3 product?
I haven’t heard from Precise in a while…I will add them on to the list though – you are right.
NOT to be intentionally confrontational, but how can you claim this is the “COMPLETE LIST” of anything?
IMHO, a here is a GLARING omission from the list:
Here is a mobile, ON-DEVICE end-user performance measuring tool. NOT a simulator (like many other companies who shall go nameless), it actually resides on an actual mobile device, NOT Smoke-and-mirrors, but real-time performance measuring, in addition to a variety of other features & benefits for a mobile USER: 3P Mobile; Privacy, Performance & Personalization are the 3 pillars of a mobile users experience!
My apologies, I read your initial post but DIDN’T comprehend!
You can add SolarWinds “Synthetic End User Monitor” to your list. We used the trial version and it seems easy enough to get started with for all your web based needs.
Thank you for your list. One of the challenges with “End User Experience” monitoring is the misuse of the term “end-user experience.” While doing research, I’ve been finding that most solutions monitor traffic (from end-users) once it hits the datacenter, which really is not an accurate reflection of the *true* experience the end user has while using business applications on their computing device (desktop/laptop/mobile). The disparity between the datacenter perspective and the *true* end user experience seems to be getting even broader with the increasing complexity of app and network architectures. Does anyone else see this as a growing issue?
Exactly.. I keep spending time researching an ‘end-user experience’ application/device only to see that it is either like a network sniffer with very unfriendly UI, or it has a nice UI but lacks the ability to drill down into user sessions.
I have deployed the Gomez RUM product (formerly Vantage Agentless or Adlex) at two sites. It does report Real User experience by sniffing the packets at the datacentre. It is difficult to explain this in a short blog such as this – but deep packet inspection can detect when the client has received all components by the TCP/IP acks that are sent back from the client. The only part of client latency not reflecting is page rendering time by the browser once all the packets have arrived – but this is tiny.
I loved the Adlex product. However – at that time it did not handle Ajax well (or at all) – but this was a few years ago. There are technical issues with handling AJAX – so I am not sure if Compuware solved it. Other solutions use Script Injection (Javascript) in the web page – in a similar fashion to Google Analytics.
Hi,
very interesting list.
Please, consider also ip-label (French competitor) offer (cloudobserver), a RUM with powerful OLAP reporting,
allowing people make deep analysis.
More informations at http://www.ip-label.co.uk/index.php/en/End-User-Monitoring-QoE-Solutions/Cloud-Observer
Atomic Labs Pion
Application Performance WebTuna
Site Confidence Real User Monitoring
Extrahop
OpTier Experience Manager
It’s a confusing list. I am curious which ones can rival Truesight (Coradiant) for being able to drill down to each user session, read cookies, see a list of every object and page downloaded, having that split into ssl, network, host, user wait etc.
Most of these just gives big picture information.. Sometimes we want to drill down to an actual users session.
Hi – nice list.
You should also include Performance GUARD (PG) from CapaSystems.
PG measures on a wider range than most other tools – network latency, response time measurements on IP level, i.e. NIC-2-NIC for client/server applications as well as through Internet Explorer for web applications.
Furthermore PG also measures the health of the end user workstation, i.e. CPU, Memory, Disk, network etc. – not only for the PC as a whole but for each individual process running on the workstation.
Even further, PG also measures the startup time and the login time from all workstations in the organisation. This will allow for detecting slow starting PC’s as well as detecting what causes the most hang time across all user…
And, of course, there is my company – AppDynamics, which has a very powerful EUM solution for .NET, Java and PHP: http://www.appdynamics.com/eum.php
Qualica technologies has a tool that uses onsite nodes to compare performance with real last mile nodes. http://www.qualica.com/performance
RUM and EUM is quite a confusing subject for many people. End User Monitoring includes a variety of techniques such as: (a) the use of synthetic transactions; (b) Datacentre appliance that sniff network packets; (c) Embedded Javascript at the Browser. I think you need to be a little clearer about the method used by each product and the Pros and Cons.
Most IT people consider EUM simply to be technique (a) shown above and are not properly aware of either (b) or (c). Most Vendors consider RUM to be techniques (b) or (c).
I too have started to research the subject and have outline the pros and cons of each method (see link below). I can’t guarantee that this information is completely accurate yet. The info is work in progress.
http://www.sde0.com/index.php/best-practice/end-user-monitoring
Hi! You’ve done a great job compiling the list.
I think, Shopping Cart Diagnostics http://www.shopping-cart-diagnostics.com would make a good addition.
This is the tool to monitor end-user experience for ecommerce websites – it detects any problems customers are facing throughout the shopping process – from login to the payment stage.
Similarly, it possesses the functionality to measure performance characteristics and a number of other essential parameters.
http://www.CloudClicks.co.uk offers synthetic transactions as a service. This removes some of the cons Doug correctly points out in sythetic monitoring.
Because it is software as a service there is no maintenance and they are cheap as this is their core business.
We at Pingdom have just opened up our new RUM service for beta testing. You can apply for beta access on our Real user monitoring info page at: https://www.pingdom.com/rum/
Hi Alon,
Did you ever create a list for robotic transactions? My two -cents is the performance monitoring industry as a whole would be more honest it would be easier to create categories for buys if the whole “end user experience” terminology was dropped. As any good dictionary will tell you “experience is subjective” so we’re not monitoring the experience…we’re at best providing a “user’s view”, point of view or a perspective. Ask yourself this, if you take a snapshot of a sunset with the world’s best camera and send it to to me are you sending me your experience of the sunset? No. Its your view of the sunset. That, in effect is the best any monitoring solution can provide, a view. The rest is marketing. It would be easier for buyers in this marketplace to parse out what technology they need if the marketing-speak was more specific.
I would like to know which of these tools require “an agent” to be installed on end users desktop. The reason I ask is if I need to install a user monitor agent on a users desktop I will be defeating the purpose. Basically if I see a user issue it could be due to the badly designed agent causing havoc with users desktop
Another thing is how can Mobile App (Which connects to the company network but not via a browser) be monitored or does it really need monitoring.?
Hopefully the above “silly questions” could be answered
Additionally I believe Synthetic transactions are a necessary evil…
Reason being any user monitoring tool cannot really monitoring what happens say when I submit an order in Siebel and it goes to downstream system to check the balance (Loose example so please pardon). But running these transactions add to the overhead of the system being monitored.