Description
We require a Senior Data Engineer with Python experience. The role requires extensive understanding and application of Python Development and building complex solutions and applications as well as extensive understanding and application of Python and data processing and transformation. Strong understanding of Microsoft SQL Server and T-SQL Development, Strong Understanding of Code Optimization, Understanding of Real-Time Data Processing and Streaming
Understanding of Apache Beam
Requirements
To enable the organization to build and support big data opensource solutions The person needs to be able to install and maintain production opensource environments to best practice standards; ensure consistency when implementing design principals and be able to manage source code efficiently using various source control technologies. The individual needs to be able to design and implement solutions using big data opensource and streaming technologies
- Effectively conceptualise, design and create high quality, custom work flows and analytics solutions.
- Develop, test, and implement big data solution designs.
- Understand client requirements and establish knowledge of data for accurate design, analysis and retrieval.
- Pull data from various data sources and combine it to store it in a datastore for analysis and retrieval.
- Collaborating with end users on standardized and best practice approaches.
- Making suggestions and enhancements on existing solutions.
- Providing regular and timely feedback to clients on design and build status
- Educating requestors as to appropriate and desirable parameters to ensure they get the information they need
- Ensure all tasks are updated on agile boards in a timely manner
- Assist Project Managers and Change and Training Managers with any project and training related administration tasks
- Actively Upskill in relevant technologies as prescribed by team leadership
- Integration and execution with Machine-learning models and AI in- flow.
- Documentation and Design of Solutions
Preferred Qualifications:
- B Degree – Computer Science / Engineering
- Hortonworks Certified
Job Related Experience:
- Minimum 4+ year’s work- experience with exposure to data pipeline development and solutions architecture as well as project management/coordination experience.
- Behavioural Competencies:
- Good Communication Skills
- Good Presentation Skills
- Good Adaptability
- Must take Initiative
- Good at Planning and organising
- Good at Teamwork
- Good at Influencing
- Good at Problem Solving
- Must have Attention to Detail
- Must be good at Analytical Thinking
- Must have a desire for Innovation
- Must be able to conceptualise ideas
Desired Skills:
- Python
- Microsoft SQL Server
- T-SQL Development
Desired Work Experience:
- 5 to 10 years
Desired Qualification Level:
- Degree
About The Employer:
Description
We require a Senior Data Engineer with Python experience. The role requires extensive understanding and application of Python Development and building complex solutions and applications as well as extensive understanding and application of Python and data processing and transformation. Strong understanding of Microsoft SQL Server and T-SQL Development, Strong Understanding of Code Optimization, Understanding of Real-Time Data Processing and Streaming
Understanding of Apache Beam
Requirements
To enable the organization to build and support big data opensource solutions The person needs to be able to install and maintain production opensource environments to best practice standards; ensure consistency when implementing design principals and be able to manage source code efficiently using various source control technologies. The individual needs to be able to design and implement solutions using big data opensource and streaming technologies
– Effectively conceptualise, design and create high quality, custom work flows and analytics solutions.
– Develop, test, and implement big data solution designs.
– Understand client requirements and establish knowledge of data for accurate design, analysis and retrieval.
– Pull data from various data sources and combine it to store it in a datastore for analysis and retrieval.
– Collaborating with end users on standardized and best practice approaches.
– Making suggestions and enhancements on existing solutions.
– Providing regular and timely feedback to clients on design and build status
– Educating requestors as to appropriate and desirable parameters to ensure they get the information they need
– Ensure all tasks are updated on agile boards in a timely manner
– Assist Project Managers and Change and Training Managers with any project and training related administration tasks
– Actively Upskill in relevant technologies as prescribed by team leadership
– Integration and execution with Machine-learning models and AI in- flow.
– Documentation and Design of Solutions
Preferred Qualifications:
– B Degree – Computer Science / Engineering
– Hortonworks Certified
Job Related Experience:
– Minimum 4+ year’s work- experience with exposure to data pipeline development and solutions architecture as well as project management/coordination experience.
– Behavioural Competencies:
– Good Communication Skills
– Good Presentation Skills
– Good Adaptability
– Must take Initiative
– Good at Planning and organising
– Good at Teamwork
– Good at Influencing
– Good at Problem Solving
– Must have Attention to Detail
– Must be good at Analytical Thinking
– Must have a desire for Innovation
– Must be able to conceptualise ideas