Our client is a leader in the retail industry with a strong focus on customer satisfaction and innovation. We are currently looking for a talented and motivated data engineer to join their dynamic team. In this role, you will have the opportunity to work on exciting projects, helping us define and solve real business problems and see solutions through to the end by using data and applying state-of-the-art machine learning operations principles. The Data Engineer is responsible for and specializes in the development of the architecture and data platform that ensures the data is ready for analytical usage, thus supporting both the data scientists and data analysts. The data engineer develops the foundation for various data operations and does analysis on the data.
REQUIREMENTS
- BSc in Mathematics, Economics, Computer Science Management or Statistics, National Diploma in Information Technology, BEng.
- 5 years of relevant experience within the field of data engineering/software development.
- Knowledge of core computing concepts like programming and algorithms to build robust data systems.
- Knowledge of application development and working of APIs
- 3- 4 years of programming
- Proficient in Visualisation
- Cloud tools: (preferably on Google Cloud (GCP) :
- Big Query, Composer/Airflow, AI Platform, Kubernetes, Dataflow, Pub/Sub, Cloud (Functions, Scheduler, SQL, Storage), Compute Engine, Monitoring
- Knowledge of the following will be advantageous:
- Git workflow, Agile management (JIRA, Shortcut, Trello), Slack, CI/CD, Docker, SQL, Python, JavaScript, Google Workspaces, Visualization tools: QlikView / / Grafana Looker Studio
JOB DESCRIPTION/DUTIES
- Data Exploration and data quality assurance
- Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality
- Acquire data from primary or secondary data sources and maintain databases/data systems
- Filter and “clean” data by reviewing other reports and performance indicators to locate and correct code problems
- Development, construction, and maintenance of data architectures.
- Conducting testing on large scale data platforms.
- Handling error logs and building robust data pipelines
- Ability to handle raw and unstructured data.
- Provide recommendations for data improvement, quality, and efficiency of data.
- Ensure and support the data architecture utilized by data scientists and analysts.
- Development of data processes for data modelling, mining, and data production
- Analyse data through descriptive statistics
- Communicating results with the team and business using data visualization
- Design and build production grade minimum viable products such as dashboards and published notebooks for business users, linked to the feature stores, scoring engines and optimisation models.
- Feature engineering
- Standardization or Scaling of the data
- Create Value Transformation like Square or Cube
- Design and build the feature stores for model training
- Operationalize feature stores for model training
- Work with management to prioritize business and information needs
- Locate and define new process improvement opportunities
- Presentations to management
- Identify the goal and frame the business problem.
- Gather information on resources, constraints, assumptions, risks.
- Prepare Analytical Goals and Benefit to business
- Understanding business and information flow
- Drive best practice, continuous improvement, and innovation within the Data Engineering function
- Evaluate new technologies and products to determine feasibility and desirability of incorporating their capabilities into data engineering
General:
- Only shortlisted candidates will be contacted. Should you not hear from us after 30 days you may consider your application unsuccessful
- In keeping with our client’s employment equity requirements, only South African citizens will be considered.
- Please include your current salary and salary expectations.
Desired Skills:
- APIs
- Cloud Computing
- Data
- Data Engineering
- Programming
- Python