Strands of information scattered throughout the organisation may have little value but, when consolidated, they often provide insights that lead to more informed decisions about vital strategic issues.
This is the view of Michael de Andrade, MD of information solutions specialist, EnterpriseWorx.
“The first step in consolidating disparate data sources and integrating the data is to organise the data environment,” he says.
“That means getting back to basics. The enterprise’s over-arching data architecture must to be structured to avoid a situation where different parts of the organisation operate in isolation. Instead, the data architecture must be able to draw together all aspects of the business.
“In customer relationship management, for example, databases may have been developed separately for marketing, sales and finance. Each of these aspects can work independently, since the key responsibilities of the individuals in each ‘silo’ are different.
"However, management needs to be able to view overall performance. Data must be brought together from these disparate sources and consolidated so it can be accessed from a single location.”
The sheer volume of data is compounded by the complexity of data sources. There are three main issues to consider: customisable data versus "black box" data; unstructured data versus structured data; and data behind the organisation’s firewall versus the cloud.
Many companies, having grown through merger and acquisition activity, struggle with all kinds of legacy systems, different and inconsistent processes, and dissimilar data structures.
“Sometimes these include ‘black-box’ systems that contain code that is difficult to customise and manipulate,” says De Andrade.
“Software applications then need to be developed to provide links to newer, more customisable systems.”
Unstructured data – covering anything from corporate e-mail through text messaging, social networking and blogs to wikis – poses different challenges.
“The trick lies in identifying which content has value to a specific audience,” says De Andrade. “When you’ve decided what you want to report on, a decision must be made on where to store it and how to manipulate it to gather meaningful information.
“Amazon’s book basket analysis, which advises customers that ‘People who bought that book also bought… ‘, is a prime example of an intelligent software application that brings a multitude of data together, generating revenue for the organisation and benefits for the customer.
“One of the main risks is that unstructured data simply gets buried, in which case it becomes redundant. Unstructured data needs to have some form of link to the organisation as a whole, and must be plugged into the overall data architecture. If you have the right technology, such as a data warehouse appliance, you can query data with a limited amount of structure and get meaningful answers.”
Cloud-based data storage poses another set of challenges. Data behind the firewall must be linked with data in the cloud.
“Once again it comes back to data architecture,” says De Andrade. “If the foundation has been laid correctly, it is possible to integrate the data from these two different data sources.
“Speed of access is an issue. It takes too long to pull data from the cloud into a database. You need to be able to quickly pump the data into another engine, such as a data warehouse appliance, and you need the right communications infrastructure and bandwidth – at an affordable price.
"Finally, there is security. Every organisation has its predefined internal security requirements. These must be aligned and/or integrated with those of the cloud-based offering for it to be possible to combine data from both sources.
“Integration strategies should be designed so as to help the right information get to the right people at the right time. Business users do not always know what they are looking for and how to trigger the query process. They should be allowed to set criteria for what they want to see, and be able to get the results they want in near real-time for faster, more informed decision making.”