Our client is going through a massive business transition that presents a unique career opportunity for a talented data professional. A strategic decision was taken for the entire information/Data platform to be re-built and re-defined, onto the AWS environment. The data mesh concept has been around for a while in the information world. Unlocking the information value within organisations is probably the key driver in the information driven world. As engineer in the core platform team, a strong team member that can support the tech lead is needed. Someone that can guide other teams integrating into the platform. A thought leader that can also challenge existing linier thinking as a fully integrated environment will not be able to function end to end if the core integration is not sound. Our client finds themselves in a place where legacy applications are being migrated so understanding the landscape, having a futuristic view and helping to define the best way forward, is a critical [Email Address Removed] ideal Data Engineer will navigate their journey, whilst keeping the plethora of new technologies top of mind. SOLID and STRONG Data Engineering experience is a MUST.
- Your primary duty is to build & maintain the data platform in AWS.
- Write data engineering code and tests using Python and SQL, using a variety of libraries and runtime environments.
- Write infrastructure as code using Terraform and build our deployment pipelines in GitLab.
- Perform code reviews, work with teammates on merge requests, participate in design sessions, etc
- Participate in the team’s Scrum processes and ceremonies.
- Engage with stakeholders to elicit requirements, demonstrate functionality, and drive a data-driven mindset in the organization.
- Operate the data platform and monitor its performance.
- Good database design skills and an understanding of various data modelling techniques and approaches.
- An ability to engineer cloud-first data solutions in the Amazon Web Services
- An understanding of the software development process, with proficiency in Git for version control and build & deployment pipelines in GitLab (or similar) using containerization (Docker).
- Ability to engineer metadata-driven approaches for sustainability and and scalability
- Experience with infrastructure as code, ideally Terraform and Terragrunt.
- An understanding of and exposure to different database technologies
- Bachelor of Science in Informatics or applicable IT qualification
Desired Skills:
- Python
- SQL
- GitLab
- AWS
- Docker
- metadata
- legacy applications