Our client is looking for a Snr Data Engineer to design data across the full range from fast/clean ETL streams through to automation.
Key Responsibilities:
Design, build and operationalize large-scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with Third Parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, [URL Removed] RedShift, Lambda, DeltaLake, Python.
Analyze, re-architect and re-platform on-premises data warehouses to data platforms on AWS cloud using AWS or Third-Party services and Kafka CC.
Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, PySpark, Scala, Kafka CC.
Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
Additional Key Responsibilities:
Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL, AWS big data technologies and Kafka CC.
Creation and support of real-time data pipelines built on AWS technologies including Glue, Lambda, Step Functions, PySpark, Athena and Kafka CC.
Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency.
Working closely with team members to drive real-time model implementations for monitoring and alerting of risk systems.
Collaborate with other tech teams to implement advanced analytics algorithms that exploit the company’s rich datasets for statistical analysis, prediction, clustering, and machine learning.
Desired Skills:
- SQL
- Kafka
- AWS
Desired Work Experience:
- 5 to 10 years Data Analysis / Data Warehousing
Desired Qualification Level:
- Degree
About The Employer:
Knowledge is our currency and our wealth. The advantage of knowing is entrenched in our organisational DNA and is demonstrated in the way we engage with our customers, business partners and stakeholders. It is our continuous pursuit to pioneer new ways to guide people towards their financial freedom.