Explore Data Science Academy has added Data Engineering to its online training offerings.
The 12-month course will provide applicants with the specific skill set required to become data engineers – a key role in the successful digital transformation of a business.
Shaun Dippnall, CEO of Explore, explains: “The use of vast quantities of data by data scientists now requires those with a specialised skill set to organise data and ensure its quality, security and availability to the organisation. These are data engineers.”
According to Dippnall apart from the skill set, there is a shortage of training courses online that offer both the content and practical application of data engineering skills required in the work place. This is particularly true of the South African market.
Course attendees will learn are SQL basics, Python programming, data modelling and cloud computing.
To these will be added specific data architecture skills such as: big data concepts, data warehousing and lakes, and data automation.
“In the digital world of today data engineering has become a specific job category as the amount of data produced by businesses engaged in digital transformation, increases exponentially,” Dippnall says.
“Today’s businesses produce new types of data in vast quantities. While data scientists have traditionally been expected to build the necessary infrastructure and data pipelines to do their work, the volume and speed of data production has resulted in the roles of data scientist and data engineer becoming separate and distinct,” he says.
“It is now recognized that companies need both data scientists and data engineers in an advanced analytics team. Naturally there is frequent collaboration between the two roles but their skills sets and tools remain different”, he adds.
Essentially, data engineers design and build pipelines that transform and transport data into a format so that, by the time it reaches the data scientists, it is usable. These pipelines take data from numerous sources and collect them into a single warehouse that presents the data uniformly as a single source.
The importance of data engineering is underscored by the fact that two of the world’s largest cloud computing hyper scalers – Microsoft and Amazon – have recently established data centres in South Africa. “These critical developments by two of the world’s largest corporations underline just how important the data warehousing is in the modern digital environment. Customers can now store their data in South Africa with the assurance that they can retain complete ownership of it,” says Dippnall.
The Explore Data Engineering course requires applicants to have a basic analytical background and will involve 10 hours of study a week.