We are searching for an experienced Senior Data Engineer to join us in driving significance together.
Primary Duties and Responsibilities:
The Data Engineer will have a knack for data analysis, data manipulation and data modelling. The consultant will be responsible for understanding and driving the overall technical vision and planning of a client’s organisation and translating business needs into technical strategy.
Required Qualifications:
– Tertiary degree, diploma or certificate in a related field (BSc Computer Science, B.IT or Informatics related degrees).
– DP-203: Data Engineering on Microsoft Azure Certification.
Experience and Knowledge:
– 8-10 years’ (5 – 8 years for mid) working experience as a Data Engineer / Database Developer.
– Experience in data mining, large scale data modelling and business requirements gathering/analysis.
– Understanding and working experience in data integration and transformation.
– Experience implementing data modelling methodologies like Dimensional Modeling and / or Data Vault.
– Working knowledge of data quality processes and master data management.
– Experience implementing design support systems using Database Management Systems (DBMS) such as SQL Server or Oracle.
– Proficiency in designing and implementing data integration and ETL solutions using SSIS, Azure Data Factory and / or SQL Server stored procedures.
– Understanding of several Big Data technologies like Hadoop, MapReduce and Spark as well as event processing or message ingestion services like Kafka, Event Hub and Stream Analytics.
– Experience in database query languages such as T-SQL, ANSI SQL, PL/SQL.
– Some experience developing software solutions using Visual Basic, C++, C#, Java or Python.
– Experience using SQL Server management Studio and Visual Studio.
– Experience implementing solutions using Azure SQL databases, Azure Synapse (Previously SQL Data Warehouse), Azure Storage Accounts (Data Lake) and / or Databricks.
– Analytical mind and business acumen
– Additional skills in the following will be taken into consideration: Tableau, Power BI, strong math skills (e.g. statistics, algebra), Scala, Python or R.
Key Responsibilities:
– Identify valuable data sources and automate collection processes.
– Undertake pre-processing of structured and unstructured data.
– Analyze large amounts of information to discover trends and patterns.
– Data Modelling (Relational and Star Schema).
– Database design.
– Database development.
– Data Warehouse Design – Build and Development.
– Database Administration.
– Database Performance Tuning and Optimisation.
– Present information using data visualization techniques.
– Propose solutions and strategies to business challenges.
– Collaborate with engineering and product development teams.
– An understanding and hands on experience on Hadoop/Spark based distributed storage and computing
frameworks.
– Real-Time analytics and batch processing.
– Strong experience in architecting analytical applications in cloud environment such as Amazon Web Services and
Microsoft Azure.
Competencies:
– Critical Thinking: Using logic and reasoning to identify the strengths and weaknesses of alternative solutions,
conclusions or approaches to problems.
– Active Learning: Understanding the implications of new information for both current and future problem-solving
and decision-making.
– Systems Analysis: Determining how a system should work and how changes in conditions, operations, and the
environment will affect outcomes.
– Complex Problem Solving: Identifying complex problems and reviewing related information to develop and
evaluate options and implement solutions.
– Deductive Reasoning: The ability to apply general rules to specific problems to produce answers that make sense.
– Inductive Reasoning: The ability to combine pieces of information to form general rules or conclusions (includes
finding a relationship among seemingly unrelated events).
– Excellent communication skills: Ability to engage with C-level stakeholders, both verbal and non-verbal and
communicate a deep understanding of the business and a broad knowledge of technology and applications.
– Technical Literacy: Possess a high level of technical literacy, which helps them determine how a software solution
fits into an organization’s current structure and assists in the development of specifications and requirements.
– Analytical Assessment: A high level of analysis to examine current systems and determine overall project needs
and scope.
– Schedule Management: Extensive time management skills to determine development schedules and milestones
and ensure that deliverables are completed on time for oneself and your team.
– Team Leadership: To oversee and direct development teams throughout the project development lifecycle,
experience with team leadership and motivation is essential.
– Ability to translate strategy and strategic objectives into measurable and executable projects.
– Experience working on large project(s) incorporating processes and procedures and standards.
Desired Skills:
- Systems Analysis
- Data Engineer
- Database Management