There will be substantial shifts in data platform technologies and methodologies over the next few years, writes Paul Morgan, chief technology officer of Decision Inc.
A number of trends are set to change the data landscape, with significant impact on organisations.
Now is the time to make decisions which can prepare the business for the future.
Over the past 20 years the industry has seen the adoption of columnar tables, in-memory databases, data warehouse appliances, agile and rapid marts, data lakes and Hadoop, and this continuum is unlikely to stop any time soon.
Organisations need to be aware of upcoming trends to ensure that the plans they put in place now to improve or replace their existing data platforms will take these shifts into account. Not only will it save them time and money, but it will put them in the lead when it comes to making the most out of their data over the long term.
One of the leading predictions is the need to blend data from both enterprise on-premise and cloud sources as well as from departmental and personal sources. This can include anything from Salesforce.com to AC Nielsen to Excel combined into one source point. There is also going to be a push towards analysing new data sources before they’ve been incorporated into a formal data warehouse, and for faster, near real-time performance from analytics solutions.
Complex and time-consuming processes such as cube loading are likely to become far simpler. It is also likely that hybrid environments of on-premise and cloud-based server solutions will grow in popularity and prevalence, while the separate worlds of business intelligence (BI) and advanced analytics will merge. Decision makers are going to expect that both traditional historical data as well as derived or forecasted data will be included in one solution.
Amidst this change will also come increased pressure on the IT department to limit the HR costs which are commonly associated with the maintenance of a complex infrastructure and technical processes. In short, repeatedly stating that data will grow exponentially is meaningless, but preparing for a tomorrow which is far more demanding and dynamic definitely is not.
