DATA ENGINEER Role
We are looking forData Engineer Professionals with 5+ years solid development experience in Data Engineering and has a solid knowledge base of the SDLC.
About The Employer:
Role tasks:
Applies advanced knowledge of area:
• Managing projects / processes
• Ability to develop within a specific sought-after programming language
• Strong working knowledge with software development tools, techniques and approaches used to build application solutions
• Working knowledge with software development tools, techniques and approaches used to build application solutions
• Cloud computing technologies, Business drivers and emerging computing trends
Generic Technical / Functional skills
• Understanding of integration between different technologies
• Assisting with the business case
• Coordination between development and support environments
• Planning and monitoring
• Eliciting requirements
• Requirements organisation
• Translating and simplifying requirements
• Requirements management and communication
• Requirements analysis
• Document requirements in appropriate format depending on methodology followed
• Assist with identification and management of risks
Specific Technical / Functional skills
• Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark.
• Experience with building data pipelines and applications to stream and process datasets at low latencies.
• Show efficiency in handling data – tracking data lineage, ensuring data quality, and improving discoverability of data.
• Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning of high-level data structures.
• Experience designing and supporting large-scale distributed systems in a production environment
Beneficial:
• Certification: AWS Certified Developer Associate / Solutions Architect
• Bitbucket / Git • Jira / Confluence • Familiar with data streaming services such as Apache Kafka, Amazon Kinesis, or similar tools
• CI / CD Tool (Nexus / Jenkins).
Soft skills:
• Ability and willingness to coach and give training to fellow colleagues and users when required