Exciting opportunities for Data Engineers at a large fast paced, high tech company looking to grow their team in Cape Town for either perm or contract roles with competitive salaries.
As a Data Engineer you will be tasked to simplify access to the services and resources, governing and maintaining the platform as well as design and implement data pipelines and systems to enable clean and usable data to manage and maintain the underlying platform that supports all teams.
Key responsibilities:
- Ensuring that data systems meet the company’s regulatory commitments, and that data is reliable and efficient
- Support best business capabilities for high performance database solutioning
- Actively participate in team, cross-discipline and vendor-driven collaboration sessions or forums to increase understanding of the working environment by contribution and participation in the relevant Data Governance Frameworks
- Partner with Technology, Business and other stakeholders to ensure that database structures, programmes and applications meet business delivery requirements.
- Design and implement scalable end-to-end database solutions including:
- address issues of data migration i.e. validation, clean-up and mapping and consistently apply data dictionaries
- Data cleanse using data bricks
- component design and development
- Data governance and data management frameworks
- BI and Warehouse design
In depth knowledge and understanding of:
- Data governance and data management frameworks
- BI and Warehouse design
- Movement of data into the Cloud using the following tools Apache Kafka, Storm, Flume for ingesting data or Amazon Web Services (AWS) Cloud Development Kit (CDK)
- Processing data and ensuring the efficient moving of that desperate “Data Lake” data for future analysis and visualization.
- Operating with real-time streams, data warehouse queries, JSON, CSV, raw data is a daily occurrence
Key Skills:
- Cloud Computing
- Azure, AWS, PaaS, Saas
- Visualization
- PowerBI / Tableau
- Analysis
- Exploratory Data Analysis – EDA
- Automation
- Scripting Data Pipelines for scheduled movement of data
Additional Requirements:
- SQL
- MS SQL server
- NoSQL
- C#, Json- calling APIs
- Python
- PowerShell
- Apache Spark
- Kafka
- Scala
Desired Skills:
- AWS
- azure
- Tableau
- Power BI