Operational complexity in virtualized, scale-out, and cloud environments and composite Web-based applications will drive demand for automated analytic performance management and optimization tools that can quickly discover, filter, correlate, remediate, and ideally prevent performance and availability slowdowns, outages, and other service-interrupting incidents.

The need to rapidly sort through tens of thousands — or even hundreds of thousands — of monitor variables, alerts and events to quickly discover problems and pinpoint root causes far exceeds the capabilities of manual methods.

To meet this growing need, IDC expects powerful performance management tools, based on sophisticated statistical analysis and modeling techniques, to emerge from niche status and become a recognized mainstream technology during the coming year. These analytics will be particularly important in driving increased demand for application performance management (APM) and end user experience monitoring tools that can provide a real-time end-to-end view of the health and business impact of the total environment.

Typically, IT infrastructure devices, applications, and IT-based business processes are monitored to see how they are performing. Monitored metrics are tested against thresholds (often adaptive ones) to see if they are exceeding defined limits or service objectives.

With the proliferation of scale-out architectures, virtual machines, and public and private clouds for applications deployment, the number of monitored elements increases rapidly and often results in a large stream of data with many variables that must be quickly scanned and analyzed to discover problems and find root causes. Multivariate statistical analysis and modeling are long-established mathematical techniques for analyzing large volumes of data, discovering meaningful relationships between variables, and building formulas that can be used to predict how related variables will behave in the future.

What is emerging is the wider application of this methodology, often called predictive analytics, to discovering, predicting, analyzing, and even preventing IT performance and availability problems. Key use cases include application performance management, virtualization management, and cloud management.

IDC expects wider distribution and use of this technology during the coming year from a growing number of vendors given the challenges of managing today's large, complex dynamic environments.

This article originally appeared in "Worldwide System Infrastructure Software 2012 Top 10 Predictions" IDC Document # 231593, December 2011, on www.idc.com.

About Tim Grieser

Tim Grieser is Program Vice President, Enterprise System Management Software, at IDC. He has extensive background in system management software technology including the use of predictive models for performance management and capacity planning.

Time Grieser's email

Twitter: @TimGrieser