Role Profile: Azure Data Engineer with PowerBi
Position Summary
The Data Engineer is responsible for leading the design, development, and maintenance of data pipelines and data infrastructure. This role involves overseeing a team of data engineers, ensuring high-quality data solutions, and collaborating with stakeholders to align data engineering efforts with business goals. This role plays a crucial role in architecting scalable data systems, optimizing data workflows, and ensuring data integrity and performance.
Key Responsibilities
- Leadership & Team Management
- Lead and mentor a team of data engineers, providing guidance on best practices, technical solutions, and career development.
- Manage project timelines, allocate resources, and ensure the team meets deliverables.
- Facilitate knowledge sharing and foster a collaborative team environment.
- Data Architecture & Design
- Architect and design scalable and efficient data systems and pipelines.
- Develop data models and schema designs to support data integration and analytics.
- Ensure that data infrastructure is designed for scalability, performance, and reliability.
- Data Pipeline Development
- Oversee the development and implementation of ETL (Extract, Transform, Load) processes.
- Optimize data workflows and pipelines for performance and efficiency.
- Integrate data from various sources, including databases, APIs, and cloud services.
- Technical Expertise & Best Practices
- Stay up-to-date with industry trends and emerging technologies in data engineering.
- Implement and advocate for best practices in data engineering, including code quality, documentation, and testing.
- Troubleshoot and resolve complex data issues and system failures.
- Collaboration & Stakeholder Engagement
- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and ensure alignment.
- Collaborate with IT and DevOps teams to manage data infrastructure and deployment processes.
- Communicate technical concepts and data-related insights to non-technical stakeholders.
- Data Quality & Governance
- Ensure data quality, consistency, and accuracy across data systems.
- Implement data governance and security policies to protect sensitive information.
- Monitor data systems for performance, reliability, and compliance with regulatory requirements.
Qualifications
- Education
- Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees or certifications (e.g., Azure Certified Data Engineer – Specialty, Google Cloud Professional Data Engineer) are advantageous.
- Experience
- Extensive experience as a Data Engineer with a proven track record of managing complex data projects.
- Demonstrated experience in leading and mentoring technical teams.
- Hands-on experience with data engineering tools and technologies
- Skills
- Strong proficiency in programming languages relevant to data engineering (e.g., Python, Java, SQL).
- Experience with cloud platforms (e.g, Azure) and their data services.
- Data Modeling & Design
- Design and implement data models that meet the business requirements.
- Develop ETL (Extract, Transform, Load) processes to integrate data from various sources.
- Ensure data accuracy, consistency, and integrity.
- BI Solution Development:
- Create interactive dashboards, reports, and visualizations using BI tools (e.g., Power BI)
- Develop and maintain SQL queries, stored procedures, and database objects.
- Implement data warehousing solutions and OLAP cubes as needed.
- Deep understanding of data warehousing solutions and data modeling techniques.
- Excellent problem-solving, analytical, and debugging skills.
- Personal Attributes
- Strong leadership and communication skills, with the ability to lead a team and collaborate effectively with stakeholders.
- Detail-oriented with a focus on delivering high-quality and reliable data solutions.
- Adaptable and able to thrive in a fast-paced, evolving environment.
Working Conditions
- Environment
- Typically hybrid
- May require occasional travel for team meetings, conferences, or client engagements.
- Tools & Technology
- Proficiency in data engineering and data processing tools and platforms.
- Familiarity with version control systems (e.g., Git) and CI/CD pipelines.
Desired Skills:
- Azure Cloud
Desired Work Experience:
- 5 to 10 years
Desired Qualification Level:
- Degree