Responsibilities:
– Lead a team of 4 subordinates
– Build and support data pipelines and datamarts
– Help facilitate gathering data from a variety of different sources
– Ensure data quality standards are maintained
– Responsible for the infrastructure that provides insights from raw data
– Handle large volumes of data in batch and real time
– Developing proof concepts and implementing complex data solutions
– Code, test and document new or modified data systems
– Build automation tools
– Create data catalogue
– Interpret data results to business customers
– Develop standards and processes for integration projects and initiatives
– Provide technical assistance to junior team members
Requirements:
– Bachelor’s in Computer Science/ Computer Engineering
– 5 years data engineering in an agile environment
– Required skills in: AWS(EMR, EC2, S3) Python, PySpark (mandatory) or Spark, BI Data Modelling and SQL
– 3 – 5 years team lead experience
– Must have broad experience in a range of software development and programming fields
– Technical data analysis and database design experience
– Expert knowledge of data modelling
– Experience in using Big Data batch and streaming tools