Data-driven applications are helping drive cloud growth, according to a big data survey of more than 300 US IT decision makers conducted by Unravel Data. The data also reveals that enterprises are most concerned with a lack of sufficient technical talent to properly manage these data systems as well as the perceived high cost of deploying a modern data infrastructure.
Overall, the survey reveals that enterprises are overwhelmingly embracing the cloud, with data applications being a major driver. According to the survey, 79 percent of the respondents have big data workloads currently running in the cloud, while 83 percent have a strategy to move existing big data applications into the cloud.
From a data perspective, the dominance of the cloud isn't surprising. The cloud is an ideal place to deploy data workloads due to its elastic compute requirements. For example, consider e-commerce retailers and the spiking traffic they experience on a day like Cyber Monday. With apps in the cloud, they have instant access to additional compute and storage resources to accommodate traffic surges and to scale back down during quieter times. In a traditional on-premises setting, they would have to provision extra servers and other infrastructure that would go unused most of the year.
The survey also revealed that most respondents are embracing both a multi-cloud strategy (56 percent) and a hybrid cloud strategy (56 percent). Hybrid cloud deployments allow enterprises to keep sensitive workloads on-premises, while multi-cloud approaches allow them to cost-optimize between the various platforms.
Despite the positive news for the cloud, the survey also revealed concerns around how to technically operationalize and manage big data systems. Respondents cited a lack of technical talent to manage deployments as their top pain point (45 percent) related to big data. Big data technologies can be complex and often require Ph.D.-level expertise to manage and troubleshoot manually. DataOps teams can address this skill gap by leveraging AI and machine learning to provide automated recommendations and actions to fix bottlenecks and errors in complex data pipelines.
The second biggest pain point highlighted in the image above was cost (41 percent). There's hope here going forward as cloud and a robust approach to Data Operations and performance management reduces the cost per application, cost per query and optimizes overall operating and capital costs by fully optimizing and providing predictability to the end-to-end data infrastructure resources.
There was some good news regarding spending on big data and IT in general. Over half of all IT decision makers (57 percent) say big data projects will account for a quarter or more of their IT budgets in 2019. You can expect big data's share of that pie to grow even more as organizations scale out their data apps into full production. Overall, 61 percent expect their IT budgets to grow in 2019.
As we see consistently, enterprises have embraced a wide range of technologies to help them harness the power of data and those investments are starting to pay off, accelerated in part by the availability and maturity of public cloud and hybrid cloud offerings. However, operational challenges remain and these can be addressed through a more robust Data Operations approach to people, process and technology needs to support production workloads and applications.
Methodology: The survey was conducted by Sapio Research on behalf of Unravel in January 2019 Respondents were IT decision makers from firms with more than 1,000 employees. 35 percent of respondents held C-level or VP positions, while the remaining 65 percent were directors or managers.
