The last few years have seen a seismic shift in the way organisations do business. The pandemic triggered massive digital transformation and modernisation with a rapid move to the cloud. Very little thought was given to things like governance, responsiveness or cost, as companies were very much in survival mode.
By Dan Sommer, senior director and global market intelligence lead at Qlik
Today, sees a very different backdrop, as organisations grapple with a recessionary environment. Regulations, tech de-coupling and geopolitical worries add to the complexities they face. Now, nearly seven out of 10 global tech leaders have expressed concern about the growing technology investment required to remain competitive. Yet interestingly few, if any, want to reduce data efforts. Here we take a closer look at some of the key trends we expect to emerge in 2023.
Turbo-charge real time data and operational response
Decision Velocity describes the use of data to rapidly make informed decisions. At scale, it can shorten the data-to-action pipeline for humans – decreasing the time it takes for people to find data and increasing the frequency of acting on it.
Through the combination of analytics, automation and AI, organisations can vastly improve decision accuracy and velocity. While humans can take weeks or months to make a strategic or operational decision, machines can make thousands of decisions per second. A real asset in challenging times. Yet, whilst automating operational decision making makes sense, it’s the job of the human workforce to chaperone and curate these decisions both at the beginning and end of the automation chain.
In addition to technology, data literacy will also shorten the time to decisions. If we can shorten the time from data to action, incrementally, thousands of times a day, it will have a real impact. From a skills perspective, all of this data literacy investment will pay off as new roles emerge with a focus on innovation – roles like chief decision officer and decision designer for example.
Code optimisation: high or low?
Low-code app platforms have proved essential in times of crisis, and they are regularly cited for their ability to deliver more business applications faster and cheaper.
Yet, some organisations have programmers and developers who simply want prompts they can code in. This is particularly the case in data engineering and data science, as those fields get reinvented for the cloud. To cater to these needs, there has been an emergence of high-code tools, which provide templates for coders who want maximum flexibility. These tools not only drive the creation of apps, they also increase the consumption of data and insights.
For example, application automation enables workers to create chains of events triggered by data. AutoML gives business analysts access to the most advanced algorithms. And data transformations within data-delivery pipelines can be largely automated, too.
These two camps will always exist, but the choice should not be between low-code and high-code but on code optimisation – focusing on the highest productivity and best business outcomes given the available skill sets.
Data storytelling at the heart of the action
For decades the mantra for organisations has been ‘get the right amount of information at the right time, to the right people.’ Today, this is more important than ever. Yet, when data is stored in so many different places it’s extremely tough to do. Fortunately, you don’t have to get all the data to all the people all the time. Having the right slices of small data at the right time is more useful and impactful. Not every insight has to be arrived at through user exploration. Many can be more prescriptive and recommendation oriented, delivered straight from the data.
Data storytelling is being touted as the way to get data to make more sense to users. It reaches us on a different emotional level and compels us to act. However, data storytelling needs to be so much more than just adding charts to an infographic or power point presentation. Reporting, alerting, automations, embedding, and predictions, are mechanisms that makes storytelling more impactful through connecting to action.
Market consolidation and new opportunities
A large telecommunications company I recently spoke to revealed that, before they embarked on their transformation programme, they were only using five percent of their data.
Many companies are in the same situation and it’s not by choice. Part of the problem is fragmentation and disjointed systems with little or no integration. There’s lots of overhead and very slow time to value. In this increasingly fragmented world, we are actually seeing a trend in the opposite direction: convergence.
We’re seeing the consolidation of previously siloed systems including data integration, management, analytics AI, visualisation, data science and automation – opening up a world of opportunity for cross-pollination.
Combining these functions makes it easier for data producers and consumers to collaborate, starting with the product, outcomes, or decisions they have in mind and working backward to build agile data pipelines around their business goals. Common standards and APIs enable interoperability. When a vendor operates across more segments, convergence is even easier. This isn’t about going ‘all-in’ on one data stack, which can lead to vendor lock-in or compromise compliance. Instead, it’s about choosing platforms that can work with multiple stacks, and consolidating the data across them.
An old new world
During the pandemic organisations quickly modernised applications and moved data to the cloud. This has created a Wild West of start-ups, who often dub themselves part of ‘the modern data stack’ fuelled by venture capital (VC), each going after one specialisation.
While winners will emerge, the vast majority will disappear as industries mature and consolidate. This trend will accelerate as VC funding goes from boom to bust. We can expect a big wave of mergers and acquisitions. It happened in the on-premise world, and it will happen again in the cloud.
From a cost perspective, it’s not sustainable for organisations to work with a wide array of niche vendors. Fortunately, many of the features will be recreated in the larger integrated data and analytics platforms. As cloud markets mature, managers may abandon architectures reliant on too many start-ups that struggle. Instead, these start-ups may be used as a source to alleviate the developer skills shortage.
Data is all organisations’ silver lining – as a smarter approach to data can help organisations navigate crisis – enabling professionals to mine data and insights, trigger action and anticipate what is coming down the line.