Phyton Proficient
PySpark – Proficient
SQL Competent
Solution Architecture Competent
API Design – Competent
Containers Competent
CI/CD Competent
Azure Cloud – Competent
Data Stream patterns and technology Proficient
Data stream patterns and technology Proficient
Data engineering design patterns Competent
Mining data Beneficialy
An undergraduate qualification (Bachelor’s degree or equivalent) in the relevant IM discipline and/or Technical competencies and certification with relevant years of experience in a similar role.
Role-specific knowledge:
- Data Lake
- Data Modeling
- Data Architecture
- Azure Data Environment
Specialist Areas:
Unstructured Data – Applies to /wiki/spaces/DAGDG/pages/[Phone Number Removed]; and any products that handles large scale data such as images.
Strong experience of building large scale file shipping and pipelines, ideally using Azure services such as AzCopy and Azure Data Lake. Experience of managing unstructured file meta-data, conversions, standardisation and related workflows. Experience of building analysis jobs that scale on technologies such as Databricks or Azure Batch.
Safety Knowledge:
Provides a consistent outstanding role model concerning safety practices with a deep understanding of the importance of safety