This popular article from the BSMdigest archives was originally published in 2006. Although not everyone quoted in the article still works at those companies, much of the content still holds true today ...

Nothing more clearly indicates whether a service is succeeding or failing than the end-user experience -- and yet most companies do not have the monitoring technology in place to really know what is happening to their customers online right now.

“When end-user monitoring tools are correctly configured, they provide the most direct indication of the performance of the business service,” says Rich Schofield of Dimension Data.

Monitoring the end-user experience is a key element of BSM. By keeping track of the end-users that IT ultimately supports, in addition to the IT devices and applications themselves, IT management can be aligned more tightly with the business goals of the company.

Why Monitor End-Users?

“Ultimately what matters is the success or failure of the consumer to conduct business,” explains Geoff Galat of Tealeaf.

Recently TeaLeaf, a provider of online customer experience management solutions, commissioned Harris Interactive to survey consumers -- and the results were astonishing, with nearly 90% of users experiencing some type of failure in conducting a web-based transaction. According to Galat, the challenge is that the customer experience occurs in the one place businesses are typically blind to -- the end-user’s browser. End-user monitoring can shine a light on this previously unreachable area.

“Anybody doing business online needs to monitor end-users,”
Galat adds. “You would never run any kind of business without being able to observe your customers and take action based on what happens to them, whether positive or negative. The web makes that even more critical, because you are never face-to-face with that customer -- their complete interaction with you is online.”

The Missing Link

Most people will agree that understanding the end-user experience is beneficial, but ironically many companies don’t have the right tools to perform adequate end-user monitoring.

“Fault management is well accepted, but some companies need help seeing performance management as a requirement beyond fault,” explains Dr. Steve Fulton of NetQoS. “Systems that are fault tolerant still experience performance issues that require attention. Similarly, performance of the elements of the system or the system as a whole does not describe the performance delivered to end-users, which matters most.”

Since many companies still rely on the traditional form of monitoring focused on IT device performance, receiving alerts when a server or network is down or slow, they are often unaware what is actually happening at the other end.

“Organizations can often tell you the fan speed of a server, but not whether or not a customer can log into the system,” adds Tony Barbagallo of GroundWork Open Source.

“Many companies are missing the true end-user experience,” Daniel Tabor of IBM. “Whether you can get to a web server and back is not always as relevant as whether you can get to the web server and then access the backend systems that support the transaction all the way through.”

Targeting the Right Metrics

When we talk about the end-user experience, what specific metrics do we really want to focus on?

“The best indication of how the application is performing for the end-user is to measure response times,” says Dr. Fulton. “The network, server and application components of response time are the most important metrics to monitor.”

Schofield specifies that important end-user metrics include total transaction time, TCP set-up time, DNS look-up time, server processing time, time spent in a client PC/device, and time spent in the network.

For others, transaction times are not always the main focus. The bottom line is that a company must focus on the metrics that indicate user success, which can translate directly to customer satisfaction.

“Companies have been sold on the idea of customer experience online equating solely to speed of delivery, such as site response time, when in fact the performance of a website, from the consumer perspective, is made up of far more than just speeds and feeds,” said Galat. “The same Harris Interactive survey I referred to earlier identified what really matters to the user -- security, ability to complete a transaction, and receiving a confirmation upon purchase. Web page download speed was important to only 3% of the people surveyed.”

Targeting the right metrics depends on understanding your business and your customer’s needs. While there is no universal list of the right metrics to measure, information about the true user experience can enlighten companies about application performance in terms of their own customer’s requirements.

The Key to BSM

End-user experience metrics are part of the foundation of business service management. These metrics gauge whether the service is ultimately successful. Without the end-user experience, there is no context for understanding the performance of the underlying infrastructure, which means there is no way to accurately measure whether a device or application is performing below expectations. The end-user metrics define the expectations. Infrastructure monitoring is still necessary, but it must be combined with end-user monitoring to be truly effective. BSM is the correlation of user experience monitoring with infrastructure monitoring to deliver an end-to-end view of service health -– a level of visibility that was never possible before.