With the core big data trends of 2014 maturing around the globe, and new trends emerging, South African’s find ourselves in a fortunate position of being able to learn from the trials of international organisations. By examining evolving big data trends around the world, local businesses are able to deploy big data more quickly and for the right reasons, says Gary Allemann, MD, Master Data Management.
One of the main trend is a shift in strategy with regards to big data as big data projects are increasingly business, rather than IT, driven.
Our partner, US big data leaders, Datameer, are seeing an increasing amount of interest from business users with many new big data projects driven by the marketing or sales departments. This shift is driven by the reality that earlier adopters can gain a time advantage over competitors, for example around customer insight, which will be hard to erode
The conversation has shifted from “What is big data?” to “Where can we gain quick benefit?”
Leading self-service big data platforms have emerged to put the entire big data lifecycle – data preparation, data integration, analytics and visualisation – into the hands of business users, while providing controls such as security, data governance and even data quality.
Datameer also see a shift in the application of big data in the US and Europe.
Where early adopters focussed more on web analytics, more mature implementations are focussing on more sophisticated uses including the optimisation of the enterprise data warehouse, customer analytics and customer experience management, fraud prevention and compliance.
The lesson for local companies is simple. Big data is not pie in the sky – international competitors in financial services, telecommunications, healthcare, retail, education and government – are gaining new insights into their business that allow them to compete more effectively. We can look at these case studies to understand how big data can be applied in our market.
For big data to deliver value, as with any technology, it must shift from being IT driven (yet another platform) to being business driven. This reality is driving another international trend.
Big data is increasingly being deployed at a departmental level to answer specific, specialised questions. Since each department has its own set of unique business problems, an overarching enterprise solution often takes too long to deploy. Individual departments are finding that they can get the answers they need, now, by deploying their own solutions.
The introduction of Hadoop as a Service (HaaS) is contributing towards this reality, as business users can deploy quickly without relying in scarce technical resources.
In South Africa, cloud solutions for big data are at an early stage of adoption with cost, bandwidth and privacy all raised as legitimate concerns. However, as data governance policies mature they must cater for cloud, with appropriate due diligence and controls to manage these risks.
IT can enable departmental initiatives, while retaining control, through the provision of private (or mixed public/private) cloud Hadoop deployments, coupled with tools such as Datameer that give visibility into the full Hadoop ecosystem. This makes it easier to track and control how the data lake is growing, who has access to which data, and what it is being used for.
Big data should be a focus in any companies’ data governance strategy as companies grapple with both the opportunities and the very real challenges that big data brings.
With some big data implementations running into years, and tens of millions of dollars, the opportunity cost for local business is high. As relatively late entrants, local business can learn from international mistakes and get to value quickly.