Client based in Bryanston seeks the services of Data Engineer.
Create and maintain optimal data pipeline architecture,
– Expand and optimizing our azure data lake and data pipeline architecture, as well as optimizing data flow and collection
– Identify, design, and implement data governance process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
– Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Python and Azure Services technologies.
– Work with the Governance team and analytics experts to strive for greater functionality in our data systems.
– Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
– Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets.
– Experience with Azure cloud services: ADF, Databricks, Data Lake et.
– Experience with object-oriented/object function scripting languages: preferably Python
Data Science/Computer Science/ Computational Mathematics/IT/Mathematics/Statistics or related technical and/ or analytical qualifications.