Python Data Engineer LWB_0313 at Mediro ICT – Gauteng Johannesburg Region

Oct 18, 2022

A company based in Midrand is looking for a Python Data Engineerto join their team on a long-term contract until Dec 2025. CV to [Email Address Removed]

  • Drive and safeguard the ingest of logistic data into the cloud data hub
  • Drive and safeguard the Business Object provisioning across all relevant data lake layers
  • Develop and operate necessary features for the data provisioning
  • Run and operate all necessary cloud components for the data provisioning
  • Safeguarding, improving, and automating all necessary Aws cloud components
  • Technical Safeguarding of all changes / used components concerning the ITLBHM AWS components across all participants (Hub, AG Data Engineer Department, 3rd party Supplier,) from code changes (Pull Request Reviews) to component and architecture changes
  • Maintenance and function enhance the Data provisioning for the ITLBHM Business Object provisioning
  • Drive and solve error analysis during data ingest and data provisioning for ITLBHM Business Objects

Minimum Requirements

Formal qualifications:

  • Master’s degree in Computer Science, Software Engineering, or a related field or similar qualification.
  • 3 years’ experience as a Data Engineer
  • 3 years’ experience as a Python Data Engineer
  • 2 years’ experience on AWS

Role-specific knowledge:

  • At least 3 years of experience as a data engineer
  • 3 years’ experience in Aws service for data engineers
    • ECS (Elastic Container Service)
    • Athena
    • Lambda
    • Glue
  • 3 years’ experience with Databases for Data Engineers e.g., AWS DynamoDB (NO-SQL)
  • 3 years’ experience in Python (Python 3x) and Pyspark for writing Spark applications
  • 3 years’ experience with scripting (Powershell, Bash)
  • Basic experience with AWS components such as
    • VPC / Iam
    • CludWatch (Metrics and Logs)
    • Param Store
    • Secret Manager
    • SNS
    • S3
    • Kinesis Streams
  • 1 years’ experience in data modeling (Oracle SQL)
  • Analytical skills in analyzing large and complex data sets.
  • Business Intelligence (BI) Experience (Advantageous)
  • Data extraction and data preparation for tableau (Advantageous)
  • Experience with Streaming (e.g., Kafka) (Advantageous)
  • Technical data modeling and schema design (“not drag and drop”) (Advantageous)
  • Agile experience (e.g., scrum) (Advantageous)

Learn more/Apply for this position