Some organisations have been quick to recognise the value of cloud services and to better visualise the return on investment following a migration, they have long used key performance indicators (KPIs).
By Nurani Parasuraman, principal customer solutions at AWS, and Mark Schwartz, business strategist at AWS
However, measuring and tracking the effectiveness of cloud adoption initiatives requires a certain amount of expertise and it is important to know the challenges and best practices to best measure progress. It is also important to understand and interpret the KPIs so that they can be used for decision making and to learn how to avoid the pitfalls.
Two principles are essential to successful cloud adoption. First, it is important to choose the right KPIs to track progress towards desired goals. Second, operating in the cloud requires organisations to rethink and adapt their on-premises processes to take full advantage of the cloud’s capabilities.
Common challenges in tracking cloud adoption
The cloud is not a strategy in and of itself; rather, it is a remarkably powerful tool for achieving business results. A common mistake is to view cloud adoption as simply a ‘technology’ initiative, when the key objectives are often to improve business agility, operational resilience, staff productivity and reduce costs.
In this case, any attempt to measure the progress of cloud adoption needs to go further than simple IT operational metrics and needs to be linked to the company’s key business objectives. The question then arises as to whether the business objectives are clearly known and unambiguous.
Unfortunately, it is common to find that executives have strong but conflicting views on cloud adoption objectives. In addition, every business is complex, unique and the pace of transformation and maturity varies.
The cloud has led to a change in business practices, therefore, progress metrics must also change to align with this evolution. Take the example of an IT infrastructure team, whose responsiveness is measured by how quickly it resolves tickets raised by application teams to provision the infrastructure. In an on-premises environment, a common measure is the number of unresolved tickets in their queue.
In the cloud, we strive to enable application teams to provision infrastructure on their own (self-service) without the need to issue tickets. Therefore, we measure efficiency by tracking the number of automated and self-service enabled processes. Here are some best practices to consider when tracking the effectiveness of cloud adoption.
* Choose the right KPIs: When used in conjunction with effective strategic planning, KPIs help organisations understand their performance and make adjustments when necessary. The KPIs that determine the success of cloud adoption are those that measure whether the goals you set for cloud adoption have been met. The challenge today is not to find more KPIs, but rather to choose the ones that are most relevant and useful to the business. More often than not, a single KPI does not give a complete picture of performance. For example, if your goal is to improve system availability, simply tracking uptime is not enough. You will also need to measure the number of times the system is offline. KPIs do not have to be perfect at the outset and do not have to be exhaustive. Start by defining KPIs related to the most important objectives, then refine and expand them over time. Finally, to measure progress, you need a baseline that shows your current level of performance. Without this frame of reference, measuring progress is useless.
* Focus on measuring business value, not just technology metrics: We need to move from process-based metrics to outcome-based metrics. For example, it is far more important to measure the business impact (missed customer transactions, impact of failures on revenue or lost employee hours) of system downtime than to measure only uptime or downtime hours. However, it is also essential to measure the activities that drive performance, not just the ‘results’. Good KPIs measure how you are progressing towards your business goal, but they are not themselves the business goals. Goodhart’s Law, named after the British economist Charles Goodhart, is often stated as follows: “When a measure becomes an objective, it ceases to be a good measure’. This law explains that when a measure is used as a performance indicator, it inevitably ceases to function as such, as people start to manipulate it.
* Rethinking old metrics in the cloud: Because of the possibilities offered by the cloud, we need to rethink how we measure progress. For example, in the traditional model, system stability is usually measured reactively, for instance. by tracking the number of incidents after they occur. In the cloud, the opposite is true: with the cloud’s advanced monitoring and instrumentation tools, the focus is on the number of incidents that have been ‘proactively prevented’, encouraging teams to adopt the right behaviour to reduce downtime.
* Leveraging the capabilities of the cloud. Monitoring cloud adoption requires a data-driven approach. The cloud comes with tools, automation and dashboards to collect performance data, which in an on-premises world requires a significant investment. In addition to the ease of data collection, the cloud provides tools to obtain information with little effort, which can be used to measure and maintain performance.
The cloud also allows you to be more ambitious. Gone are the days when infrastructure had to be planned years in advance with room for the unexpected and the architecture had to be “future-proofed” to support unforeseen changes.
In the cloud, the infrastructure can grow or shrink automatically to accommodate changing business needs. With this flexibility you can afford to set goals that are just on the edge of being impossible. Even if you don’t reach them, you will still have learned and grown in the process and created a new dynamic.