When it comes to big data, South Africa is keeping up with the rest of the world, writes Johan Scheepers, Commvault systems engineering director for MESAT.
However, one of the largest concerns for local CIOs is the management of this data. Rising technologies, such as machine to machine (M2M) communications and the Internet of Things (IoT), are generating massive volumes of data – so much that even large organisations and municipalities are grappling with the challenges of how to manage this data.
South African enterprises and government are rapidly embracing the benefits of cost saving and better management that mining and analysing big data can offer. Big data is a veritable gold mine of information, providing insight into consumers, future trends, how to manage utility supply and improve service delivery, and even how to predict and prevent equipment failures.
User information and historical data present a host of useful knowledge which can be leveraged to promote happy customers and fully functioning, always on systems. Airlines are analysing data to balance delivering on their consumers’ needs while ensuring aircraft are properly – and predictively – maintained. Rail service providers are using big data to determine failure trends and proactively maintain rail infrastructure and locomotives.
On the income generating side of things, data mined through analysing consumer behaviour and patterns allow enterprises to streamline their sales process and boost profits. This also promotes cost saving as less money is wasted targeting uninterested markets or providing products that there is little or no market for.
On the management side of things, historical data can be used to identify patterns of failure and predict when failures may occur in future. This allows businesses to plan better, which translates to better management. They can budget for maintenance of equipment as well as to minimise the costly loss of reputation and income that failures can cause. They can also predict their resource requirements for various times of the year or future events, and plan accordingly.
The challenge isn’t what big data can do for a business though, but how to manage it so that the business can achieve what it needs to from big data. There is simply so much data being generated, that previously simple tasks such as disaster recovery (DR) become incredibly complex.
Many of the platforms in place for DR are unable to cope with the sheer volumes of data coming through on a daily basis, and data storage is becoming ever more expensive as more and more data streams in, so creative ways need to be investigated in order to manage data flow and backup. There are a number of questions that need to evaluated, the answers of which all play a significant part in managing the costs of data management.
Questions such as what data is valuable and what isn’t; how long will the data be valuable and, if it isn’t valuable now, will it be valuable in the future? Is the data valuable enough to warrant off site storage for DR purposes? If so, what kind of protection is needed for the data, and how does one determine what data requires what level of security? Then, there is also the growth curve to consider to determine future DR requirements.
Many companies are simply throwing money at the problem, seeing the need for data analytics and storage yet they are still ending up with the same complex problems of managing their data, and answering the aforementioned questions. They need a management platform that can protect their data and create DR without duplicating data and adding to the complexity and cost of data management.
It is important to have access to a DR platform that doesn’t simply duplicate data for storage, however, that offers the benefits of a data management system too. Businesses should be able to access their data at any time, yet also remain secure in the knowledge that their data is selectively and appropriately protected, available for analysis wherever the most benefit is offered, and is also available for recovery should the need arise. Basically, businesses need a platform that enables their data to work for them to the maximum benefit of the organisation.
