9th BIT Consulting, specialists in implementing SOA solutions, middleware integration and products, has been appointed as an African partner for StreamSets, a global leader in data performance management.
StreamSets offers a set of products designed to address the growing challenge of managing data in motion, in a world marked by constant data change. The company enables simpler, safer and more accurate data management, irrespective of the data source, data processing infrastructure, destination and changes to the data itself as it is moved from one source to another – a problem called data drift.
Barry de Waal, chief executive: strategy and sales at 9th BIT Consulting, says that in a world of big data, 9th BIT Consulting focuses on providing organisations with access to their data in near real time, no matter the source, format or destination.
“One of the main problems with big data is the challenge of getting data from various sources, some legacy and some current, and collecting it into a data lake (a common data store), from where it can be easily accessed,” explains De Waal. “StreamSets addresses this by focusing on the areas of data Ingestion and data drift.”
Data Ingestion is similar to data extraction, transformation and loading (ETL) processes of the past. StreamSets is the next evolution in ETL, allowing organisations to ingest data into their data lake faster and with more accuracy. This leads to more productive data processing and analysis.
StreamSets has developed connectors into many data storage formats, from traditional relational systems to current NoSQL databases, Hadoop and more. Using a graphical drag-and-drop interface, users can set up data ingestion processes with ease and with the confidence the data will be transferred accurately from any source to any destination.
“StreamSets also provides an open API to allow customers to develop their own connectors if necessary,” adds De Waal. “9th BIT has already developed a few connectors and donated them to the open source community using the API.”
Data Drift is the name given to the constantly changing nature of data. When using older ETL tools, organisations had to follow intensive processes to identify and resolve this drift. StreamSets provides the tools necessary to cater for data drift with less human intervention than in the past and less with coding.
The deal between StreamSets and 9th BIT was signed at the end of 2016, but De Waal says the company is only going to market now that its technical staff have been trained and are ready to support its customers in their data performance management projects.
