Companies face numerous challenges in leveraging enterprise data for analytics, with data portability, time-to-value, and self-service for business users as top requirements to address these challenges.
This is among the findings of an IDG Research MarketPulse survey, “Gaining Time, Savings, and Insights via Cloud-Powered Data Transformation” released by Matillon and IDG Research.
The survey polled more than 200 IT, data science, and data engineering professionals at North American organizations with at least 1 000 employees.
The top takeaways include:
Businesses waste too much time wrangling and preparing data
* It takes about a week, on average, to aggregate and prep data so that it is useful for analysis.
* Nearly half (45%) of time spent on data analytics projects is dedicated to data preparation, instead of on more strategic, high-value tasks.
* Six in 10 cited lack of scalability and flexibility as a challenge when preparing data for analytics.
* Nearly all (97%) are searching for ways to accelerate the data transformation process.
Enterprises deal with widespread operational and technical challenges in getting data analytics projects to production
* Nearly half (47%) of respondents said data control issues are the biggest challenge to data analytics projects.
* Other top challenges indicated are lack of a scalable, reliable technology platform to process large data sets (45%), having too many manual processes (38%), and challenges cleansing and preparing data (36%).
* 40% cited lack of visibility and control of data silos as a challenge when collaborating with business users.
Blending cloud data platforms presents new opportunities
* More than one-third (38%) are already using cloud data warehouses (CDWs). Long term, 43% expected to have all of their data in the cloud, with the remainder planning to pursue hybrid models that leverage both cloud and on-premises data warehouses.
* While the use of CDWs is already widespread, only 16% currently use data lakes. More than half (56%) plan to use data lakes in the future, and another 26% are considering doing so
– 57% will leverage a hybrid cloud strategy (on premises and cloud) for data management, while 22% are planning a multi-cloud strategy, and 21% will use a single cloud provider to manage all their cloud-based data.
Enterprises require data portability, time-to-value, and self-service for business users to overcome current challenges
* Respondents said data portability (57%), ease of onboarding (57%), and cost effectiveness (52%) are the top features of an analytics platform that can help them move past current obstacles.
* IT professionals favored user-friendly (50%) and easier connections to data sources (50%) as top features.
* Data professionals (data engineers, data scientists, data architects, etc) cited time-to-value (57%) and ability to provide self-service (51%) as important capabilities.
“Over the course of several surveys, IDG has identified the key challenges organisations face preparing massive data sets for analytics engines,” says Tom Schmidt, digital content director at IDG’s Strategic Marketing Services. “The latest study finds that powerful and scalable cloud-native solutions can help optimize this business-critical process.”
Matthew Scullion, CEO of Matillon, adds: “There is clearly still a struggle for enterprises to perform data transformation faster to provide the business with analytics-ready insights.
“This research highlights not only what holds them back, but more importantly, the capabilities and features needed from a solution to deliver business value through transformed data. The power and the economics of cloud-native solutions help companies achieve faster time to value that can make a big impact in creating a data-driven culture for IT and business users.”