|
ENVIRONMENT:
A rapidly growing E-Commerce Platform seeks a highly talented Data Engineer to make an invaluable contribution to data democratisation and literacy vision by making accessible and easy-to-use data products and tools. You will require a Comp-Sci Degree or 3 years’ relevant industry experience, MySQL, PostgreSQL, Java, Python, in-depth understanding of OLAP, Data Marts, Star Scheme, Snowflake, be familiar with Jenkins, Travis, Circle CI, Docker, Kubernetes, experience with Kafka, Pub/Sub, Stream data pipeline frameworks or solutions such as Apache Flink, Apache Beam, Storm, Databricks & significant technical experience and a proven track record of Data Modelling, Schema Design, Data Warehousing & Data Lake design and implementation and Lambda/Kappa architectures.
DUTIES:
- Design, develop, test and maintain data architecture.
- Prepare data for descriptive, predictive and prescriptive modelling.
- Automate repetitive tasks and manual processes related with the data usage.
- Optimize data delivery.
- Design, develop and test large stream data pipelines to ingest, aggregate, clean, and distribute data models ready for analysis.
- Ensure the highest standard in data integrity.
- Leverage best practices in continuous integration and delivery.
- Collaborate with other engineers, ML experts, analysts, and stakeholders to produce the most efficient and valuable solutions.
- Implement features, technology, and processes that move us towards industry best practices, improving on scalability, efficiency, reliability, and security.
- Operations and ownership of systems in production, responding to incidents.
REQUIREMENTS:
Qualifications –
- Comp-Sci Degree or 3 years relevant industry experience.
Experience/Skills –
- Open source relational database systems e.g. MySQL, PostgreSQL, etc.
- Significant technical experience and a proven track record of Data Modelling, Schema Design, Data Warehousing, and Data Lake design and implementation and Lambda/Kappa architectures.
- A thorough understanding of database and data warehousing principles e.g. OLAP, Data Marts, Star Schema, Snowflake, etc.
- Java and Python.
- Familiar with CI/CD tools such as Jenkins, Travis, Circle CI, etc.
- Experience with Kafka, Pub/Sub, or other event-based systems.
- Stream data pipeline frameworks or solutions such as Apache Flink, Apache Beam, Storm, Databricks, etc.
- Working in cloud environments and with containerisation frameworks, tools and platforms (e.g. Docker, Kubernetes, GKE, etc).
- A deep understanding of data pipelining, streaming, and Big Data technologies, methods, patterns, and techniques.
- Troubleshooting complex database operations and performance issues.
- Automating tasks using shell scripting or writing small applications.
ATTRIBUTES:
- Works well with people and is passionate about helping people be their best.
- A team player, an active listener, mentor, and able to communicate well.
- Shows solid reasoning and decision making, with the ability to work under pressure.
- Is passionate about technology, systems and data.
- Is curious, always learning, and keeping up to date with the industry.
While we would really like to respond to every application, should you not be contacted for this position within 10 working days please consider your application unsuccessful.
COMMENTS:
When applying for jobs, ensure that you have the minimum job requirements. OnlySA Citizens will be considered for this role. If you are not in the mentioned location of any of the jobs, please note your relocation plans in all applications for jobs and correspondence. Please e-mail a word copy of your CV to (email address) and mention the reference numbers of the jobs. We have a list of jobs on http://www.datafin.com. Datafin IT Recruitment – Cape Town Jobs.
|
