Our client, a leading global manufacturing company seeks to employ a Data Engineer, to join their business systems team.
This role is a remote working based role, with certain days on site in Cape Town, when required.
This is a pivotal role in building and operationalizing the inclusive data necessary for the enterprise data and analytics initiatives following industry standard practices. The bulk of the work will be in building, managing and optimizing data pipelines and moving these data pipelines effectively into production for key data and analytics consumers like business/data analysts, data scientists or any employee that needs curated data for data and analytics use cases across the enterprise.
Furthermore, the function will guarantee compliance with data governance and data security requirements while creating, improving and operationalizing these integrated and reusable data pipelines. This would enable faster data access, integrated data reuse and vastly improved time-to-solution for data and analytics initiatives.
Requirements for consideration:
- You will have a degree in Computer Science, IT or similar field (Statistics, Mathematics), coupled with 2-5 years experience in foundational knowledge of Data Management practices
- In addition you will have solid and proven experience with various data management architectures like Data Warehouse, Master Data Management, Data Lake, Data Hub and the supporting processes like Data Integration, Governance, Metadata Management
- You will display a strong ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management.
- Strong experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional data integration technologies. These should include ETL/ELT, data replication/CDC, bonus: message-oriented data movement
- Basic experience in working with data governance/data quality and data security teams and specifically information stewards and privacy and security officers in moving data pipelines into production with appropriate data quality, governance and security standards and certification.
- Ability to build quick prototypes and to translate prototypes into data products and services in a diverse ecosystem
- Demonstrated success in working with large, heterogeneous datasets to extract business value using popular data preparation tools such as SSIS-ADF and (or) Databricks
- Strong experience with popular database programming languages including SQL, PL/SQL
- Experience in working with Spark Ecosystem (i.e. Databriks)
- Ability to automate pipeline development
- Experience in working with DevOps capabilities like version control, automated builds, testing and release management capabilities
- Ability to collaborate with technical and business personas
- Demonstrated success in working with both IT and business while integrating analytics and data science output into business processes and workflows.
- Basic experience working with popular data discovery, analytics and BI software tools like SSRS, Power BI or Cognos
- Exposure to hybrid deployments: Cloud and On-premise, etc
- Adapt in agile methodologies and capable of applying Data-Ops principles to data pipelines to improve the communication, integration, reuse and automation of data flows between data managers and consumers across an organization
Due to the exceptionally high volumes of applications, we are unable to correspond with each applicant personally, therefore only relevant and suitable candidates will be contacted. Should you not receive any correspondence from our company within 30 days, we regret your application has been unsuccessful.
Should you meet the criteria and wish to apply, kind forward your CV, salary package details and notice period