Our Client is an well known Retail Giant Based in Brackenfell are currently looking for a Senior Data Engineer to join their dynamic team. They are know throughout South Africa for their great prices and deals.
Minimum Qualification Required:
IT Degree/Diploma
AWS Certification at least to associate level
The succesful candidate will be responsible for:
- Creating data feeds from on-premise to AWS Cloud Support data feeds in production on break fix basis
- Creating data marts using Talend or similar ETL development tool
- Manipulating data using python and pyspark
- Processing data using the Hadoop paradigm particularly using EMR, AWS’s distribution of Hadoop
- Devop for Big Data and Business Intelligence including automated testing and deployment
- Design and develop data feeds from an on-premise environment into a datalake environment in an AWS cloud environment
- Design and develop programmatic transformations of the solution, by correctly partitioning, formatting and validating the data quality
- Design and develop programmatic transformation, combinations and calculations to populate complex datamarts based on feed from the datalake
- Provide operational support to datamart datafeeds and datamarts
- Design infrastructure required to develop and operate datalake data feeds
- Design infrastructure required to develop and operate datamarts, their user interfaces and the feeds required to populate the datalake.
Minimum Years of Experience:
3 – 5 Years:
- Business Intelligence
- Retail Operations
- Big Data
- Extract Transform and Load (ETL) processes
- Cloud AWS
- Agile exposure, Kanban or Scrum
Skills Required
- Talend
- AWS: EMR, EC2, S3
- Python
- PySpark or Spark
- Business Intelligence Data modelling
- SQL
For more information on this great opportunity, apply now
Desired Skills:
- Talend
- AWS
- EMR
- EC2
- S3
- Python
- PySpark
- Business Intelligence
- Data Modeling
- SQL
- retail Operations
- Big Data
- ETL
- Extract Transform and Load
- Agile
- Kanban
- Hadoop
- Data engineering
Desired Work Experience:
- 5 to 10 years