Data Engineer
Our client is seeking a skilled Big Data Architect to design, implement, and maintain robust architectures for handling large datasets. Your role will involve developing data pipelines, ensuring data quality, and collaborating with data analysts to meet business needs.
R600 000 – R700 000
Key Responsibilities:
- Design and build systems for extracting value from massive datasets.
- Develop and implement data pipelines for efficient data movement and processing (ETL).
- Ensure data quality through validation and cleaning techniques.
- Build and manage scalable data storage and processing systems using distributed frameworks like Apache Hadoop.
- Collaborate with data analysts to understand data requirements and translate them into technical solutions.
- Implement and maintain data security measures to ensure privacy and compliance with governance policies.
- Monitor and troubleshoot data pipelines for optimal performance.
Qualifications:
- Degree in Information Systems, Computer Science, Engineering, Communications, or equivalent.
- Experience in databases; certification is advantageous.
- Experience with Big Data frameworks; certification is advantageous.
- Proficiency in a scripting language.
- Experience working with cloud data architectures; certification is advantageous.
Technology Skills:
- SQL Databases: MySQL, PostgreSQL, Oracle
- Programming Languages: Python, PySpark, Scala
- Big Data Processing Frameworks: Apache Hadoop, Spark, Flink
- Distributed Storage: HDFS, Cloud Object Stores
- Data Warehousing & Modeling: Apache Hive, Cloud data warehouses
- ETL: Apache NiFi
- Streaming Frameworks: Apache Kafka
- Work Flow Orchestration: Apache Airflow, Oozie
- Cloud Platforms: AWS, Azure, GCP
- Scripting Languages: Bash Scripting
As Specialist Recruiters for professionals in your industry, we are well geared to represent your best career interests. Whether you are an active job seeker or just browsing, let’s have a no stress conversation about your next career move! It’s always good to have a great recruiter looking out for you!
For more exciting positions visit our website [URL Removed] or Call us on [Phone Number Removed]; and quote this advert.
Please note if you have not received feedback within two weeks, please consider your application unsuccessful for this particular opportunity.
Desired Skills:
- AWS
- Cloud Platforms
- Azure
- Bash Scripting