A global data engineering business is on the look out for a Senior Data Engineer to join their growing team. They understand the complexities of transforming data and they help guide customers through their data journey while offering them comfort and simplicity on their digital transformation journey
You will be working on fixed-scope, fixed-timeline projects across AWS and GCP for large national and international customers, including banks, airlines, medical aid providers and telecommunications companies
If you want to be part of this business that transforms Data into value and modernises infrastructures, APPLY TODAY!!!!
Required Experience:
- Hands-on experience in building ETL pipelines on Big Data, using modern cloud Data Engineering tools (6+ years):
- AWS Glue (highly preferred)
- GCP DataFlow (preferred)
- Azure Data Factory
- Object Oriented Python programming (5+ years)
- Python package development
- Python class development
- Pythonic standards across the board
- Working knowledge of PySpark (preferred) or Apache Beam
- Efficient use of the PySpark DataFrame API
- Knowledge of big data file formats (Parquet, ORC, AVRO) and partitioning strategies
- Hands-on experience in writing SQL queries (5+ years)
- ANSI-SQL (preferred)
- Postgresql (preferred)
- MS-SQL
- MySQL
- Oracle
- Database experience
- Amazon Redshift
- Google BigQuery
- Or equivalent
- Infrastructure as code:
- Terraform (preferred)
- Cloudformation
- AWS CDK
- Hands on experience in making use of additional cloud services:
- AWS Lambda (highly important)
- AWS IAM (highly important)
- Cross account roles
- IAM policies
- Amazon S3 (highly important)
- AWS Step Functions (important)
- Amazon SQS
- Amazon SNS
- Amazon DynamoDB
- Amazon Kinesis is a plus
- Ability to scope a Data Solution end-to-end
- Service integrations
- Drawing architectural diagrams
- Scalability, security, fault-tolerance
Required Qualifications:
- BSc Computer Science/Engineering (or equivalent experience)
- AWS or GCP Certifications (Associate and above)
Reference Number for this position is GZ55908 which is a permanent position offering up to [URL Removed] per annum negotiable on experience and ability. Contact Garth on [Email Address Removed] or call him on [Phone Number Removed]; to discuss this and other opportunities.
Are you ready for a change of scenery? The e-Merge IT recruitment is a specialist niche recruitment agency. We offer our candidates options so that we can successfully place the right developers with the right companies in the right roles. Check out the e-Merge website [URL Removed] for more great positions.
Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!
Desired Skills:
- AWS Glue
- GCO DataFlow
- Azure Data Factory
- Python
- SQL
- PySpark
Desired Qualification Level:
- Degree