Industry experts offer thoughtful, insightful, and often controversial predictions on how APM, AIOps, Observability, OpenTelemetry and related technologies will evolve and impact business in 2024. Part 3 covers even more on Observability.

Start with: 2024 Application Performance Management Predictions - Part 1

Start with: 2024 Application Performance Management Predictions - Part 2

Observability Recognized as a Data Problem

Despite pouring $17 billion into observability and monitoring tools each year, enterprises are seeing a negligible impact on mean-time-to-resolution (MTTR) — in fact it is increasing. Why? Modern distributed applications are complex, they change multiple times a day which leads to DevOps teams seeing "unknown" problems in production every day. When troubleshooting an "unknown" problem, DevOps teams must triangulate on data points to determine where the problem may be occurring. That's where the problems start. Some data points are in a logging tool, a monitoring tool, or an APM tool. The best practice is often to screenshot what each tool is showing and post in a Slack channel so the final decision maker can correlate. This is not sustainable. For Observability to deliver on its promise, the observability data must be in one place, not in several silos. If the data is in one place it's easier to navigate, find relevant context for the incident being investigated, and for the DevOps team to collaborate in one consistent interface (that's not Slack!).
Jeremy Burton
CEO, Observe

TELEMETRY DATA AS AN ENTERPRISE ASSET

As organizations scale, telemetry data complexity, volume, and cost of data management will continue to increase. Observability, security, and business teams will demand access to a wider variety of data. More functional groups will consider incorporating AI/ML into their observability workflows, but data quality will constrain business outcomes. As organizations realize that it's not the ML models but underlying data that will separate them from their competitors, they will have to take a newer enterprise approach to manage their telemetry data. In the coming year, we will see organizations taking steps to manage telemetry data as an enterprise asset and putting mechanisms and governance in place, first to understand the data and then provision the right data in suitable formats, with appropriate quality and context to observability and security teams.
Ajay Khanna
CMO, Mezmo

DATA TIERING FOR OBSERVABILITY

Data Tiering will be widely adopted with buy-in from Engineer to Executive. It's not sustainable to store all your observability data in a premium storage tier, like a log data index. This is evidenced by the fact that 82% of DevOps and SRE teams limit log ingestion as a mechanism to reduce costs often or all the time. Moving forward, teams will leverage a combination of best-of-breed tools to move their data to the optimal location based on their use case (real-time analytics, ad hoc queries, compliance, etc.) and desired cost efficiencies (hot and searchable index, object storage, etc.).
Ozan Unlu
CEO, Edge Delta

OBSERVABILITY CONVERGES WITH SECURITY

An emerging trend in 2023 — one that is sure to grow in 2024 — is the need for shared visibility into key enabling apps and IT infrastructure technologies from both an operational and security standpoint. Kubernetes is a primary example. No matter what model or teams you support internally, there's a shared interest in the performance and security of technologies like Kubernetes that are so fundamental to modern apps and infrastructure. Regardless of whether security and observability will converge in terms of organizational structure and roles and responsibilities of teams, there does seem to be a big advantage to adding security capabilities to the traditional three pillars of logs, metrics and traces for observability.
Asaf Yigal
Co-Founder and CTO, Logz.io

OBSERVABILTY TRAINING TO COMBAT STAFF SHORTAGE

In the past few years, IT has gotten increasingly more complex and interconnected. The array of tools required to manage the sprawl of Digital Experience Management tools has also expanded, as have the alerts that require the attention of short-staffed IT teams. The complexity of managing the sprawl is biting into productivity and profitability. We have heard IT leaders say slow-running systems and applications, as well as outdated technology, are directly impacting the growth and performance of their organization. One of the key challenges for IT leaders who are driving digital transformation and improving DEM is being understaffed or having employees who lack key skills. To address this challenge, we will see more IT leaders set aside budget to retrain IT staff and focus on Unified Observability technology with greater automation to help close the skills gap.
Mike Marks
VP Product Marketing, Riverbed

OBSERVABILITY FOR FINOPS

Observability will move up the organization to support the sustainability and FinOps drive. The combined pressure of needing to adopt more sustainable practices and tackle rising cloud costs will catapult observability from an IT priority to a business requirement in 2024. On the one hand, organizations will face the dualling trend of seeing their use of AI increasing cloud costs and adding to scope 3 emissions, while also — when applied to observability data — providing insights into how carbon footprint and expenditure can be reduced to offset this impact. By applying AI in this way, organizations will mature their emerging FinOps practices, by identifying hotspots of inefficiency in their cloud environment to enable more effective cost optimization. As an added advantage, this increased use of observability will make it easier for organizations to not just report on their ESG goals but gain the granular insights they need to continuously identify and eliminate hotspots where resources are being inefficiently consumed. This in turn enables them to orchestrate systems for optimal resource utilization, reducing emissions as well as the cost of running their cloud environments. We will therefore see growing interest in use cases of observability beyond the IT department as the wider business begins to take note.
Bernd Greifeneder
CTO and Founder, Dynatrace

COST OF OBSERVABILITY INCREASES

The costs of observability will swell with data explosion and bloat, so find observability solutions that offer innovative ways to reduce the costs of data storage. One problem with running cloud-native applications on multi-cloud, multi-region architectures is that they generate enormous amounts of observability data — and increasingly so as organizations scale. With the volume of observability data exploding, most organizations aren't equipped to handle the budget-breaking costs that ensue. We can beat this observability bloat by investing in smart data collection and data hygiene. Many organizations just collect everything they can get their hands on. This costs a lot of money, creates noisy clutter in the environment, and hampers the ability to search and gain insights. Instead, observability solutions should help organizations filter out unneeded data to reduce costs and also help customers move their less valued data into far less expensive storage repositories without losing the ability to actively query it when needed.
Asaf Yigal
Co-Founder and CTO, Logz.io

FOCUS ON COST OPTIMIZATION FOR OBSERVABILITY

Buyers of observability tools will continue to become more cost conscious. Tools that adjust the amount of observability data collected based on the health of the underlying services will become powerful solutions for organizations who want to control their observability spend without increasing risk.
Will Krause
VP of Engineering, Circonus

The year 2024 will continue to proudly carry the banner of efficiency and cost optimization. The main thing organizations focused in 2023 were the cloud providers, which in most cases are the top bill for any cloud-based company. Next in line are observability solutions which in some cases add up to 25% on top of the cloud provider expenses. Teams will turn to more cost effective solutions for log management, metrics and full tracing which are continuing to grow in data volumes. We'll see SMBs taking wiser decisions into which observability solution to adopt based on future projected costs and adopting multi-year deals that guardrail expenses, understanding the magnitude of their potential switching costs.
Shahar Azulay
CEO and Co-Founder, groundcover

BOLSTERING OBSERVABILITY ROI: TYING OPERATIONAL STATUS TO BUSINESS OUTCOMES

Enterprise IT leaders struggle with rising observability costs. As custom metadata grows to provide increased context, enterprises are often saddled with surprise billing overages. IT organizations have been trained to reduce these costs by reducing the observability footprint, but that often leaves blind spots, increasing downtime and reducing ROI. To bolster ROI, enterprise IT leaders and business observability teams need to tie operational status to business outcomes. This approach helps teams reduce the need for increased data and spending time on dashboards. Instead, they can focus on what matters for customer experience.
Gagan Singh
VP, Product Marketing, Elastic

NEW OBSERVABILITY PRICING MODELS

We'll see observability vendors offering more diverse pricing models that include paying by the queried or used data only, paying by seats or paying by infrastructure size alone, trying to offer customer alternatives to the controversial volume-based models.
Shahar Azulay
CEO and Co-Founder, groundcover

TOOL SPRAWL BECOMES A THING OF THE PAST

In 2024, tool sprawl will start to become a thing of the past. Companies often fall victim to employing an unnecessarily high number of IT tools, causing bottlenecks, inefficiencies, and negative ROI. Tech professionals are spending more time learning the intricacies of these tools than they are completing their actual work. Leaders need to mitigate this issue by consolidating their implementations — allowing for better manageability, efficiency, and security. With the constantly changing digital transformation, companies need to opt for a "best of suite" to ensure their employees can get to the real work that needs to be done.
Jaya Baloo
CSO, Rapid7

Go to: 2024 Application Performance Management Predictions - Part 4, covering OpenTelemetry.