With the likes of artificial intelligence (AI) and automation expected to take over many repetitive tasks, companies globally are trying to fill scarce skills in IT, data analytics, and even AI itself.
Advanced technologies are creating opportunities for the next-generation of data professionals who are willing to adapt and leverage their tech-mindedness in this new era of work. However, this requires a combination of hard and soft skills to supplement training and experience, according to Andreas Bartsch, head of innovation and services at PBT Group.
“Digital transformation and evolving into data-driven businesses have become priorities for organisations across industry sectors,” he says. “And this has led to an enormous demand for data- and digital skills that is proving challenging to meet, even internationally.”
A 2023 research project by Forbes Advisor indicated that 93% of UK businesses say there is an IT skills gap, fuelled by rapid advances in technology such as AI, data analytics and cloud computing. And Salesforce’s 2022 Global Digital Skills Index estimated that 14 G20 countries could miss out on $11,5-trillion in cumulative GDP growth if the digital skills gap is not addressed.
South Africa’s Higher Education Institutions are responding to the local demand for digital skills, often in collaboration with industry. Twelve of the country’s major universities offer undergraduate degrees, with postgraduate qualifications up to Doctorate level, aimed at producing specialised data professionals such as Data Engineers, Data Architects, Data Scientists and more.
As these careers are considered “emerging”, new courses are being added all the time. However, the rate at which graduates are entering the market is not meeting the rate of demand.
Technical skills remain a cornerstone
Data-related jobs have expanded exponentially in recent years, moving out of the proverbial IT basement and into almost all areas of business from finance to marketing and sales, to human resources – anywhere that data-driven decision making is increasingly business critical. Organisations rely on data professionals to manage enormous volumes of data and provide the insights necessary to meet customer demand while optimising internal processes.
Considering the somewhat technical nature of data specialist roles, when it comes to ongoing critical skills, exposure to some form of programming (anything like Python, SAS, Java, C++), is essential. Furthermore, Structured Query Language (SQL) is a programming language for storing and processing information in a relational database – and probably the “home language” of the data world. As such, irrespective of the data specialist role, it will be in one’s best interest to master SQL.
Learning how to use some of the more prevalent data engineering, cloud engineering and data analytics technologies – such as Microsoft’s Azure and PowerBI, Amazon Web Services (AWS), IBM DataStage or Informatica, and SAS – can also empower data professionals to fit more easily into any organisation where a technology stack has been selected, and most of these technology providers offer accessible training courses and certifications.
Some understanding of the Modern Data Platform concepts and associated best practice principles are also essential.
“Over and above this, it should be noted that, irrespective of the technology, a good data professional is technology and cloud agnostic,” Bartsch says.
Complementing the hard skills, Nicky Pantland, a data analyst at PBT Group, highlights the significance of soft skills for data professionals to truly thrive and make themselves marketable in the future.
The same Forbes Advisor study supports this view, ranking problem solving – which encompasses analytical and critical thinking as well as troubleshooting capabilities – as the most important soft skill for data and technology professionals.
“Equally important soft skills include continuous learning, creativity, emotional intelligence, and communication – written, verbal and presentation skills. It is not just about having the technical capabilities, but also about understanding the origins and formation of data – being curious about data – and then being able to share insights meaningfully with non-data professionals in the business,” says Pantland.
Pantland says that knowing where the organisational data is sourced from can identify potential systemic issues, limitations, or discrepancies. And then understanding the processes, definitions, and rules that moulded the data can help align it better with the current organisational objectives.
“Without understanding the ‘why’ of data lineage, there is a significant risk of misuse, leading to skewed outputs. This is where having the necessary soft skills becomes an essential complement to the hard, technical knowledge of data specialists,” says Pantland.
Pantland also emphasises the need to be creative. “This provides the foundation of having a growth mindset, and an ability to make logical decisions. Despite the technical aspects of data roles, having good interpersonal communication and showing leadership are also important.”
Bartsch and Pantland agree that the combination of hard and soft skills will define the success of data professionals in the current and future landscape. While technical skills are foundational, soft skills are what empower professionals to harness data responsibly and innovatively.
It is this blend of skills that PBT Group advocates for aspiring and current data professionals to remain relevant and excel in their careers. A holistic skill set is not merely a career asset but a necessity in the fast-evolving world of data and technology.