Financial institutions are ideally positioned to reap a host of benefits from big data, given their access to multiple streams of information, writes Gary Allemann, MD of Master Data Management.
However, most banks are still standing at the edge of the big data wave, hesitant to take the plunge. Despite their prime placement to take advantage of this phenomenon, there are three important reasons for the slow adoption of big data in South African financial services.
The confusion around defining big data
South Africa still doesn’t fully understand what big data is. This is mainly due to the – vastly confusing – variety of individual definitions, presentations and use cases presented by virtually every infrastructure vendor, mega-IT-vendor and storage vendor. Each vendor suggests that big data is what they offer. It’s little wonder we are confused.
Doug Laney coined the original “Three V’s definition” in 2001 which describes big data as a combination of the Variety of types of data, the Velocity or speed of processing the data, and the Volume of data available. Applying this definition remains a good technical test of whether or not something can be considered big data. Simply put – if you can use existing SQL type technology, then it cannot be considered big data.
My preferred definition is to define big data by its primary benefit: the ability to deliver valuable insight in business time. Big data approaches reduce development time and work well for ad hoc analysis – ensuring that answers can be delivered while they are still relevant to the business decision. This is of particular value when data sources are varied and complex – big data approaches are well suited to integrating diverse and complex data sets for analysis.
Identifying big data applications
Banks are struggling to identify suitable applications, although there are many potential use cases out there. If we look at case studies from international big data leaders, we can see that their customers have moved beyond IT experimentation to identifying business driven use cases for big data – such as fraud analytics, customer analytics and regulatory reporting.
One international trend has been the shift away from the centralised data lake to the delivery of departmental solutions.
For example, a major credit card vendor is combining transactional data, spatial data, merchant and customer data to identify and manage fraudulent transactions.
The marketing team, on the other hand is combining customer data with social shares to deliver better targeted marketing – saving money and increasing conversions.
Over the last several years, international banking and insurance clients have used the likes of Hadoop to solve numerous business problems – from fraud prediction, to churn reduction, to risk data aggregation and many more. These use cases, and many others, are well documented. Big data is still seen as a technology stack for IT experimentation rather than being used to solve specific business problems
Lack of skills
Finally, the lack of big data skills in South Africa remains a real challenge. Hadoop requires technical skills that are in short supply our market, and the rapidly evolving Hadoop ecosystem makes it difficult to keep skills up to date. One only needs to look at the evolution of data analytics on Hadoop – from technical, to self-service, to visual / iterative – to see how quickly it develops.
The right skills are imperative, not only from a user perspective, but also from a data partner perspective. I strongly believe that we need to adopt simpler tools that shield us from both the complexity and the evolution of Hadoop if we are to achieve full benefit from its use, and this is one of the reasons for our own partnership with Datameer.
This does not mean that every employee will become a data scientist. However, by reducing the complexity of data preparation, investigation and even analysis we can get more value from our data scientists. We can also deliver some level of governed self-service to allow the broader business community to take on some of the simpler analytics problems themselves.
There are multiple ways in which big data could both improve and support South African financial business, if banks and other financial institutions leverage the technologies and resources available.
South African financial institutions could look to trusted data partners to help them get the best value out of big data, and to implement it in a beneficial, profitable and productive manner, for the betterment of their business.