Monitoring and observability requirements are continuing to adapt to the rapid advances in public cloud, containers, serverless, microservices, and DevOps and CI/CD practices. As new technology and development processes become mainstream, enterprise adoption begins to increase, bringing its own set of security, scalability, and manageability needs. I sat down with Stephen Elliot, VP of Management Software and DevOps at IDC, to discuss where the market is headed, how legacy vendors will need to adapt, and how customers can get ahead of these trends to gain a competitive advantage.
What key changes are you seeing companies make to better position themselves for digital business?
Stephen Elliot: Business and technology executives are focusing on improving customer experience by transforming their internal processes and products through software innovation, a focus on digital-first, and direct-to-consumer.
Teams are relying more and more on data and powerful analytics to make faster, data-driven decisions. They are now able to collect and reuse data from across operations and the business to drive more product innovation, which in turn drives more revenue and higher profitability.
Many organizations are creating a dedicated CDO role to help accelerate the adoption of modern cloud technologies such as containers, microservices, and serverless, and DevOps and CI/CD processes.
The importance and specific responsibilities of a chief digital officer (CDO) vary from one organization to another, and can take one of several forms:
1. independent of the CIO
2. report up to the CIO
3. CIO assumes the role of CDO and brings in a new CIO
As with most transitions, organizations must maintain their legacy model to keep the proverbial “lights-on”. Meanwhile, they must invest aggressively to prepare for the future, and create the freedom to adopt new technology frameworks, new team structures, and even new cultures.
In some organizations it's more effective to have completely separate teams and executives to better focus on the specific speed and quality objectives of each group. This offers them the opportunity to invest in new tools, new processes, and new operating models that best match their team needs.
In large enterprises it can be particularly difficult to have traditional teams think differently, many folks don't necessarily want to change. Success requires a change management approach to address all components– people, process, technology, culture, and business thinking, as well as setting realistic timelines.
Large enterprises typically have hybrid and multi-cloud strategies that require different architectures from on-prem deployments. Legacy infrastructure will continue to require dedicated support, as will each public cloud deployment.
Enterprises are doing a balancing act, their traditional heritage estate was built on a foundation of risk-mitigation, but they're now being asked to move faster when selecting tools, processes, and technologies – many of which have only been around for a few short years.
What are the biggest barriers to success for these new digital initiatives?
Stephen Elliot: Companies need to find and resolve issues quicker in their business logic and their applications. They need access to data, and monitoring and observability are a huge source of valuable real-time operational data, they're a heads-up display for how business are being managed.
To resolve issues faster, modern agile organizations need 3 capabilities:
1. Ability to collect and consolidate data in real-time
2. Analyze data in real-time to find the root cause
3. Collaboration to make sure everyone is on the same page as to what the problem is and how to solve it.
By enabling teams with data access, organizations can drive more informed business decisions across both IT and line of business, including executives.
Consolidating data from traditionally disparate sources such as business, infrastructure metrics, tracing, application performance, and logs is critical to providing a comprehensive view of modern environments and how they affect applications and consumers.
Modern teams now span a wider set of responsibilities, including Developers, Infrastructure Platform Engineers, IT Operations, Site Reliability Engineers (SRE), Cloud Architects, and even business units. These teams need easy-to-use tools and preferably a single tool, to help them identify and solve problems across unified and commonly understood data. And because these groups have varying levels of technical depth, tools that either automate or augment analytics, and deliver them in a streamlined fashion – can help teams get to the right answer faster.
In addition to real-time data, analytics, and collaboration, these tools should also address the following:
■ In-depth data collection and support for high-cardinality analytics
■ Root-cause identification and directed troubleshooting
■ Auto scaling and auto remediation capabilities
■ Integration with CI/CD tool chain
How important are analytics for Monitoring and Observability solutions?
Stephen Elliot: Monitoring and APM vendors are all investing more and more into analytics, however, some are investing more heavily in R&D and have comprehensive and well-defined product roadmaps. Customers need to do their due diligence to clearly understand what capabilities each vendor actually provides. This means digging into the patents, and then seeing first-hand how real-time, machine learning, AI, and high-cardinality are incorporated into actual product capabilities.
Everyone is talking about speed and real-time, and there's no doubt that it's becoming more important as companies adopt modern application development, deployment, and management frameworks, and transitioning their applications and platforms to run on the public cloud, microservices, containers, and serverless. The speed and frequency at which applications can now be deployed is making it more important to collect and analyze performance data in real-time. It's important for organizations to understand that speed is increasingly becoming a competitive advantage for the business, and performance management is a great opportunity for organizations to extend that advantage.
