As the holiday shopping season looms ahead, and online sales are positioned to challenge or even beat in-store purchases, eCommerce is on the minds of many decision makers. To help organizations decide how to gauge their eCommerce success, APMdigest compiled a list of expert opinions on the best way to measure eCommerce performance.
Of course, business side measurements such as sales, revenue and profit are great indicators of eCommerce success or failure, but for now we are looking at the question from the IT Operations side. APMdigest asked experts from across the industry – including consultants, analysts, organizations, users and the leading vendors – for their opinions on the best way to measure eCommerce performance, in terms of applications, networks and infrastructure.
Not to say that the IT Ops perspective should not align with the business. "For any (private sector) company, there are only three things executives really care about: making money, saving money, and keeping customers happy. Any KPIs you're thinking of measuring should tie to one or more of these priorities. If you're measuring anything else, those metrics are secondary to these primary business drivers." explained Jason Bloomberg, President of Intellyx.
“As companies seek to engage customers on a massive scale during the holiday season from an eclectic mix of digital channels, IT teams need a 'single source of truth' into the impact of application performance and response times on business outcomes,” adds Pete Waterhouse, Advisor, Product Marketing, CA Technologies. “Employ methods that capture essential application performance metrics such as load, errors, traffic and latency and can then aggregate and link these to business performance indicators and KPIs. This way every team is fully aligned around shared business goals and problems.”
The business goals are the ultimate goals, and the IT Ops tools and methods to keep the technology side of eCommerce up and running are meant to support these business goals.
“Understand what opportunities you are losing. Understand the extent of these lost opportunities,” said Ivo Mägi, Co-Founder and VP Product, Plumbr. “A good monitoring solution explains the extent to which a particular performance or availability issue impacts business. Getting exposure to such insight allows product managers to prioritize development tasks based on real impact and stop relying upon rumors or gut feeling.”
So in the end, gauging eCommerce success is all about business-side measurements, but our experts show that there are many ways IT Ops can look at apps and websites, networks and infrastructure, to ensure they are meeting performance requirements that will support the business. Part 1 covers analytics and machine learning.
ANALYTICS AND MACHINE LEARNING
According to users in the IT Central Station community, the more companies go online or move more of their business to online sales, the more they need analytics to understand these purchase patterns and customer behaviors.
Russell Rothstein
Founder and CEO, IT Central Station
Fast and early anomaly detection and measurement is absolutely essential during the holiday period when IT teams have no time for length war room huddles and triage. To stay sane in all the "holiday madness" employ analytics and machine learning that identify early warning signs and performance anomalies from massive data volumes, decrease noise and false-alarms, and automatically gather evidence so teams can collectively attack problems before they irreparably damage the business.
Pete Waterhouse
Advisor, Product Marketing, CA Technologies
Large enterprise applications using microservices architecture and containers (we refer to cloud native applications or systems) have a high degree of complexity and desperately need a new generation of APM monitoring. Real-time and time-series metrics analytics, processing big data quantities is a must have, with distributed tracing for deeper troubleshooting. We see machine learning as a hot area in metrics analytics.
Michael Azoff
Principal Analyst, Ovum
Ecommerce businesses need to know in real time what is happening in vast amounts of data from multiple sources —such as ecommerce platforms, web analytics, server clusters and more. By normalizing all of your data into time series, machine learning techniques can be used to learn the normal behavior of each data metric, and thus discover when a metric is behaving abnormally. Ranking and correlation of related anomalies then can transform alert storms into unified incidents. This technique allows ecommerce companies to look beyond aggregated summaries and to dive deeply and quickly into granular issues like how a product update is affecting server performance in a particular cluster, causing a drop in revenue.
David Drai
CEO, Anodot
DASHBOARDS
One of the best ways for Dev and Ops (and everyone in between) to foster productive collaboration is by providing them all clear visibility to what's working and what's not along the digital delivery pipeline. This can be accomplished by enabling the team with a reporting and analytics dashboard fed by the team's myriad collection of test automation engines, CI/CD processes and release gates. Everyone on the DevOps team should have the same goal; delivering high-quality applications at accelerated rates. With transparency to "pipeline truth" coupled with a team-focus for success collaboration comes much easier.
Larry Johnsen
VP - Customer Success, Parasoft
When development teams ask how to measure performance of ecommerce applications to manage great user experiences, I strongly recommend that they start thinking about proactive visibility of their app's behavior. Mature teams embracing DevOps are building real time app health dashboards shared by all stakeholders — a single source of truth that can be analyzed to enable focus on the user rather than who or what is responsible for any service degradation. Ideally, the solution that feeds data into that health dashboard should be the same one used by the dev teams for pre-production testing.
Amir Rozenberg
Director of Product Management, Perfecto Mobile
Faster product life cycles and ever-complex operations cause retailers to use big data analytics to understand supply chains and product distribution to reduce costs. Many retailers know all too well the intense pressure to optimize asset utilization, budgets, performance and service quality. It's essential to gaining a competitive edge and driving better business performance. The key to utilizing data engineering platforms to increase operational efficiency is to use them to unlock insights buried in log, sensor and machine data. Furthermore, servers, plant machinery, customer-owned appliances, cell towers, energy grid infrastructure and even product logs — these are all examples of assets that generate valuable data These insights include information about trends, patterns and outliers that can improve decisions, drive better operations performance and save millions.
John Morell
Senior Director of Product Marketing, Datameer
