The Data Engineer will design, implement and maintain data pipelines that are scalable, repeatable, secure and can serve multiple users within the business. In addition they will manage all aspects of the data platform. The Data Engineer will source data, assess quality, check for accuracy prior to reporting and analysis and ensure that users can access the required data.
Bachelor’s Degree at the appropriate NQF level in the area of computer science, engineering, mathematics, statistics and/or a combination of these.
Data engineering certifications such as Python, Microsoft, AWS, Hadoop, big data and cloud infrastructure advantageous.A minimum of 5 years’ experience in data engineering.
Experience with SQL and working with large scale data etc.
Experience with distributed data processing such as Hadoop, Spark, Kafka, Hive, Nifi and Hbase is advantageous.
Experience with Cloud Data Engineering technologies is advantageous.
SAP BW ETL experience is advantageous.
Experience in operationalising data science solutions or similar product development experience in a high-scale production environment is advantageous.
Project management or consulting experience applied in cross-functional projects is advantageous.
Working knowledge of multi/hybrid cloud environments and implementations.
- Cloud Data Engineering
Employer & Job Benefits:
- Standard Benefits