AWS Data Engineer
ESSENTIAL SKILLS REQUIREMENTS:
Exceptional experience/understanding (in order of importance):
- Terraform
- Python 3x
- SQL – Oracle/PostgreSQL
- Py Spark
- Boto3
- ETL
- Docker
- Linux / Unix
- Big Data
- Powershell / Bash
- Experience in working with Enterprise Collaboration tools such as Confluence, JIRA etc.
- Experience developing technical documentation and artefacts.
- Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV etc.
- Experience working with Data Quality Tools such as Great Expectations.
- Experience developing and working with REST API’s is a bonus.
- Basic experience in Networking and troubleshooting network issues.
- Any additional responsibilities assigned In the Agile Working Model (AWM) Charter
ADVANTAGEOUS SKILLS REQUIREMENTS:
– Exceptional analytical skills analysing large and complex data sets.
– Perform thorough testing and data validation to ensure the accuracy of data transformations.
– Strong written and verbal communication skills, with precise documentation.
– Self-driven team player with ability to work independently and multi-task.
– Experience building data pipeline using AWS Glue or Data Pipeline, or similar platforms.
– Familiar with data store such as AWS S3, and AWS RDS or DynamoDB.
– Experience and solid understanding of various software design patterns.
– Experience preparing specifications from which programs will be written, designed, coded, tested and debugged.
– Strong organizational skills.
Experience / understanding of AWS Components (in order of importance):
- Glue
- CloudWatch
- SNS
- Athena
- S3
- Kinesis Streams (Kinesis, Kinesis Firehose)
- Lambda
- DynamoDB
- Step Function
- Param Store
- Code Build/Pipeline
- CloudFormation
- Technical data modelling and schema design (“not drag and drop”)
- Kafka
- AWS EMR
- Redshift
QUALIFICATIONS/EXPERIENCE NEEDED
- Relevant IT / Business / Engineering Degree
- Candidates with one or more of the certifications are preferred:
- AWS Certified Cloud Practitioner
- AWS Certified SysOps Associate
- AWS Certified Developer Associate
- AWS Certified Architect Associate
- AWS Certified Architect Professional
- Hashicorp Certified Terraform Associate
Desired Skills:
- Data Engineer
- AWS
- Python