The pandemic has required organisations to adjust business models, develop new products, and fast-track their technological capabilities virtually overnight.
By Andreas Bartsch, head of innovation and services at PBT Group
People became reliant on quality data to guide their decision-making. And with the availability of 5G, edge computing, the cloud, artificial intelligence (AI), and the Internet of Things, companies have much better access to data than ever.
The ability to make sense from all this data has become a critical success factor in digital transformation initiatives. This has seen many companies put the emphasis on establishing an AI competency to analyse data at scale.
The science of data
The potential value of AI has been well-documented and has prompted a significant focus on the role of the data scientist. Unfortunately, the latter is often misunderstood, misinterpreted, and devalued when companies offer data science ‘fast-track’ programmes, especially when the selected candidates do not have the relevant academic qualification.
In my view, chief data officers (CDOs) must be conscious of maintaining a sensible ratio of data engineers to support the appointed data scientists. For AI (and data scientists) to be successful, a supporting, well-governed modern data platform remains critical.
This entails a well-defined data architecture, complemented by the respective data model and best-practice data engineering principles and data pipelines.
Many AI initiatives fail due to the non-readiness of the underlying data platform and non-automated processes. Whilst the business value is gained from AI and the expertise of the data scientists in achieving this, the effort in collecting, preparing, managing, automating, and governing the data and associated pipelines must never be underestimated.
Overcoming the skills challenge
In his recent State of the Nation Address, President Ramaphosa highlighted the important roles the state and private sector must play in enabling job creation. Local organisations are responsible to make a concerted effort to appoint or assign local expertise before looking elsewhere.
Unfortunately, the local skills pool is severely drained as corporates, SMEs, and consulting firms are fishing from the same pond. This results in exorbitant rates which, whilst lucrative to the individual, harms the industry as it is not sustainable. Additionally, it cultivates a generation of ‘job hoppers’.
Of course, budgetary constraints remain a serious challenge. The increasing salary demands exacerbate the situation, with the understandable sensitivity to consulting fees. However, when considering the likes of recruitment efforts, management overhead, training, career paths, and bonuses, the total cost to company is comparable, or even higher, than the consultant counterpart.
Talent retention
I believe that an organisation should focus on retaining its key intellectual property (IP). When it comes to data roles, this IP relates to data scientists, business analysts, enterprise architects, and potentially enterprise data modellers.
The more technical nature of the data engineer is industry and technology independent. It is here where a partnership with a local specialist data service provider can add significant value in enabling capability and scalable capacity.
The skills shortage not only creates an opportunity but demands a concerted effort between the larger organisations and their data specialist services providers to engage and collaborate in establishing academies that bridge the step from graduation to junior data engineers in a practical manner.
This should typically entail a combination of theory, practical training, and on-the-job exposure, thereby enabling capacity and capability building at scale – so desperately needed within the South African context.