All cloud is not the answer for all BI: business critical processes should stay in-house, writes Mervyn Mooi, director at Knowledge Integration Dynamics (KID).

Recent international research shows steady growth in the number of organisations implementing cloud-based business intelligence (BI). Dresner Advisory Services research puts the growth at 6% last year, while the international Business Application Research Center (BARC) and Eckerson Group, say the rate of adoption has increased dramatically in the past few months.

Dresner’s study finds that ad hoc query, advanced visualisation, dashboards, data integration and data quality, end-user self-service and reporting are the most required cloud BI features.

But while the cloud has certainly expanded our options for BI and big data analysis, it is unlikely that this surge in uptake means companies are using the cloud for all their BI, or indeed any of their mission-critical BI.

In our experience, large enterprises holding confidential and mission critical data still prefer to keep that data on-premise and run BI and analytics on that data on-premise. BI in the cloud – particularly in the public cloud – Is not taking place as quickly as people thought it would.

Enterprises have good reason to be cautious: not only does POPI demand exceedingly high levels of vigilance around control and management of confidential data, but in some industries, the very models used to analyse the data are the result of years of evolutionary development and the business’s competitive advantage depends on them.

Insurance companies and other financial institutions, for example, have highly advanced modelling that allows them to viably offer new solutions to market or control overheads and remain competitive.

These models are so important in some sectors that many enterprises hire specialised data scientists in a bid to develop the ‘ultimate algorithm’ that will help them identify new niches and optimise efficiencies. Putting this important company IP in the cloud is a risk many companies are not yet willing to take.

In addition, effective BI requires data integration and data preparation, which has a huge processing overhead. In the case of fairly traditional BI, in which historical data is analysed, or where small pockets of data are to be used, the cloud is up to the challenge. But when BI moves to address ever-changing real time data, as in the case of a myriad sensors, air control systems or vehicle trackers, the cloud is not necessarily as elastic as vendors might like us to believe.

Huge processing capacity and pipes may be needed to churn vast amounts of data on an ongoing basis, and despite dedicated capacity and lines, cloud platforms do have their limitations and are typically shared to some extent, putting constraints on big data analysis.

When real-time data and big data are vital for business success, on-premise systems still deliver optimal results and eliminate concerns around backup, recovery and business continuity for crucial processing. From a security and asset ownership point of view, companies still prepare and franchise their data on-premise and this is unlikely to change for some years to come.

However, there is a great deal of data that is already public or not highly confidential which can safely reside in the cloud and be used for enterprise BI. A car retailer, for example, could gain very useful market insights by applying BI to cloud-based data around car models, trends and social media sentiment analysis around particular vehicles.

All the data required is already in the public domain, yet the analysis of this data could deliver insights that benefit the business, and the cloud offers an efficient and cost-effective way to do so. Cloud-based BI also offers benefits such as transferring the Enterprise Information Management (EIM) and much of the security requirement to the cloud solution provider.

Cloud BI applications using SaaS, PaaS and I/DaaS are ideal for small datasets not requiring serious, governed data quality and security. But for the foreseeable future, BI will not – and probably should not – take place in the cloud around the data that matters most to the enterprise.