Introduction
We might all be heartily sick of hearing the term ‘AI’ added to everything (in ITSM, and in fact everything in life…) at present – at least this has moved us on from ‘digital transformation’ and ‘unprecedented times’…
A strong contender to the AI hype and topic of the moment is of course ‘experience management’, plus of course XLAs (experience level agreements).
The ideals of experience management (XM) are based on sound concepts and ways of working, much of which reflects gaps and problems in the ITSM world that have long needed to be fixed.
There are various tools and services now available that provide technical and people-focussed solutions to support this – often supported through AI. Digital experience (DEX/DX) as well as sentiment analysis, where user happiness and other elements are captured and analysed.
There is also inevitably the cycle of hype and overreaction associated with this – right now we are still in the hype stage of evolution. A lot of good progress and development has been made, although there are some areas that need clarification and further discussion.
In this blog I am setting out some practical guidance on how to be successful with the XM approach, potentially using XLAs. This also identifies some issues and challenges, as well as context and reference to existing ways of working, such as SLAs.
Additional content from previous blogs is also referenced.
What do we mean by XM?
Experience management is essentially an ‘Outside In’ approach to quality, improvement and value demonstration.
Orgnisations that only carry out ‘Inside Out’ measurement – monitoring and reporting of their performance – are internally focussed and therefore unaware or indifferent to the impact of the things that they do. Their own internal measures and metrics will help them to manage their operation, but are insufficient to fully capture the result and impact of their services on people, outcomes, businesses, organisations, etc.
Feedback – whether human, digital, technical – is the key to experience management. You may have created a great new product or service, which is properly costed, well built, efficiently delivered, etc. However, if the experience of buying or receiving or using this is poor, then you may not use this company again, or worse, go public in saying this.
I recently bought a new family PC – this a really great product, although the shipping experience was so dismally poor that I will never buy from this vendor again. I told them so – gave them my feedback – and I hope that they use it. Unfortunately, they have not given me any indication that they have taken any notice – they are either unaware as a company of this (so not using effective experience management), or they don’t care.
XM involves capturing feedback from multiple sources and using this to improve and meet expectations, fix issues and engage positively with your stakeholders.
This can be different types of data – direct personal contact, digital endpoint data, customer response and advocacy (NPS, surveys, complaints), employee behaviour (absence, attrition), as well as direct feedback. The key of course is how you not only capture this content but also what is done with it – as a management practice does this lead to awareness and a call for action…?
There are many good reasons for doing this of course – business improvement, avoiding loss and failure, financial efficiency, customer retention, staff retention (employee experience), quality improvement, staff well-being, visibility of issues.
In short, organisations that don’t capture various different types of feedback and act on it, will fail – not from lack of ideas or industry, but from ignorance and lack of awareness, resulting in loss of customers, profits and reputation.
Experience Management – Barclay Rae
What are XLAs?
The feedback and ‘outside-in’ concepts mentioned are not new, neither are various forms of experience management, particularly customer XM – this has been a central strategy for the retail and hospitality industries for decades. SLAs have also been around for 30 years, mostly as targets for response and fix, plus availability times in IT operations and support.
The ‘XLA’ has emerged in recent years as a new approach.
This takes the new broad definition of service experience and builds this into a customer or user agreement. This can be measured and used to manage the supplier/customer relationship. There is an obvious reference and comparison to the ‘SLA’, which has been used to measure some parts of service delivery and provide data on performance.
The challenge for SLAs was of course that they generally did not represent the full scope of service as delivered and as providing value. SLAs certainly did not represent an outside perspective, and accordingly they are not well regarded in many quarters as they can be used as a smokescreen to a bad service.
i.e. ‘we met our SLAs’, however this was not a good experience, or did not deliver business expectations.
Watertight not watermelon ‘SLAs’. – Barclay Rae
How does XM relate to SLM and SLAs?
For many years, IT organisations have captured various types of metrics, including customer feedback of some form or another.
Many organisations also have relationship management roles and functions in place, to gather customer feedback and to use this to build demand pipelines. Feedback data exists in many places across these organisations, also beyond IT in HR and finance – all of this is useful as part of an XM approach.
Although this data exists, the focus – and often the definition of success – has remained tied to the SLA performance data. This is useful as part of the feedback approach, but only a small part of the overall picture.
Using the SLA as the only ‘proper’ measure has not been a positive or successful activity; This often gives a distorted view of the actual service experience, and in turn can show the (IT) provider as out of touch.
So, the XLA idea is really to merge the measurement activities in some form, to represent a much broader definition of the service, the experience, the performance and the outcome.
Ideally this can be summarised in a small set of summary, bundled data – 4/5/6 key areas, with some means of summing up the overall service experience and outcome. This is similar to a balanced scorecard.
The customer should only need to agree the relative importance or weighting of the different elements, rather than getting bogged down in response and fix times that are meaningless to them.
The XLA therefore represents an evolved data-rich version of the SLA – it’s an expansion and development, rather than a replacement. It also still uses a lot of the data and metrics gathered for SLAs – ideally with a weighted/balanced approach to bring together disparate measures.
This offers several positive benefits of consistency and continuity of approach from the SLA/SLM world. The challenge is that this also can be the biggest issue if the lessons of poor SLAs are not learned.
Let’s not make the same mistakes – what have we learned from SLAs?
In my opinion, SLAs did not fail because the concept was bad, even although this initially was applied to a limited set of performance metrics.
SLAs failed because (1) the approach was misunderstood, (2) and they were poorly implemented…
(1) SLAs misunderstood and presented – ‘off the internet’. There has been too much of this. The notion that SLAs are simply given or presented to users or customers, based on some spurious concept of ’best practice that I found online’, rather than meaningful consultation.
Most of the value that I have seen gained from successful SLAs (yes, they do exist) over the years has been from the consultation, the meetings, the conversations. People talking and others listening and noting things down. Discussions and negotiations and iterations where both parties are working towards a suitable model, based on emerging shard goals.
It’s a clear example of ‘co-creation’ – I.e. working together to create something that is more than simply the sum of the parts.
The whole point and value of an SLA approach is not just to find an impact/urgency/priority table and plug it in. Rather its doing the work collaboratively to define what that table might mean to each party and to tweaking it accordingly, based on some consensus and agreement.
The actual SLA is less important than the process and work done to create it.
If you find some SLA details and simply give them to the other party as a done deal, then it’s doomed to fail – it’s an SLD – a ‘service level disagreement’.
(2) SLAs over-engineered to death. The other big mistake has been to take all the content that has been created in this area – ITIL and beyond – and to apply it to the letter and in detail, without any filter on context or value.
I’ve seen way too many ridiculous sets of documentation on SLAs, written in legalese and with clauses and sub-clauses, with lots of input on terms and conditions and input from multiple surveys. In most cases these are oblivious to user or business needs but are mostly seen as a way to protect the IT department.
Its no wonder that most users are not interested in participating in this process – I still regularly hear IT people complaining about this. ‘We tried doing SLAs, but they weren’t interested…’!
Many organisations are also now railing against survey fatigue, where every department sends out feedback surveys for every interaction – death by 1000 survey questions… Some companies are already limiting or even banning this activity.
We can’t repeat these mistakes with XLAs. Unfortunately, I am starting to see some evidence that these mistakes have not been learned from. I’ve seen examples where a data collection template has been applied that no one understands or agrees to and Bingo! – we have an XLA!
More commonly I’ve seen several instances where the approach and content has been applied – or attempted to be applied – in full and as a one-size-fits-all approach. In one company this was unnecessary and unwanted and could have ruined what was a very good user/supplier relationship.
Context is vitally important here – relationship, maturity, business demand, customer appetite, IT capability (technical and human), as well as scale and business demographics.
The PESTLE and ITIL 4 Four Dimensions have never been so relevant…
What key steps and activities should we follow?
Here are some practical tips for successful XLAs
1. Go and listen to your customers.
· Observe their work and work environment.
· Talk to them, but mostly listen.
· Don’t jump into solution mode too quickly.
· Where possible go to their workplace and see what they are doing.
· Try to understand what is important to them, who, what, when, where etc.
· Consider user profiling, to identify different types of users.
· This is all useful work, regardless of where this project goes.
2. Review the level of your (XM/XLA) approach – start where you are.
· Identify your organisation’s level of appetite and capability to apply XLAs.
· Take into account the current level of service relationship and a clear view of what value the XLA approach will deliver.
· Are XLAs needed or wanted?
· Identify what level of XM can realistically and effectively be applied, build a phased approach.
· Be clear on the ‘Why’ – what is the objective of doing this..?
3. Consider why SLAs in your organisation have previously failed.
· Avoid re-creating the same issues with XLAs instead of SLAs.
· Often this has come down to how a service provider approaches and interacts with its business users.
· What steps should we avoid and what did we also do well last time?
4. Continue engaging with your users and business to build and refine your service definitions.
· Ask your users what information they would need to judge the success and quality of your services – not what you think this is.
· Ideally this will produce some ‘outcomes’ for them, rather simply activities done by IT.
· Generic targets are useful, but you need to include key criteria for targets.
· Not every aspect of XM needs an SLA/XLA target.
5. Consider the impact of survey overload on your users and customers.
· Its desirable to get data and feedback, but some organisations are now limiting the number of surveys that can be used.
· Avoid over-surveying users.
· Avoid asking too many questions that may be irrelevant or take too much time.
· Keep surveys simple, intuitive and proportionate.
6. Build up a repository of service related information.
· This will help you to define and share user needs and to respond directly to them.
· Think of the approach as a data lake/unstructured data – get it in one place.
· Some services will require a consistent approach whereas others should allow for flexibility in response and measurement. Good service is not a one-size-fits-all business.
7. Start to build a scorecard model
· Get this verified for relevance and weighting by your customers and users.
· This can be presented as a compound or balanced scorecard, or also as real time dashboards.
· Experience data should be combined with other forms of measurement, feedback and performance data to provide new levels of expectation setting and management.
EXAMPLE Scorecard model (from my current research with PeopleCert)
8. Collect and co-ordinate metrics data from various sources for assessing service quality.
· CSAT and NPS data from feedback surveys and services.
· Inputs from direct contact/BRM feedback.
· Sentiment data – user and employee feedback.
· Digital data (e.g. from DEX tools, endpoint and monitoring tools).
· Outcome data – use your user’s feedback on what’s important to them as the basis for business outcome data.
· Include financial details as relevant.
· In addition, there is your existing ITSM performance data.
· You may also add developmental areas such as innovation.
9. Don’t throw away all the good stuff you’ve built with ITSM – build upon it.
· It is preferable to use XM as part of service management, not as a separate entity.
· XLAs and experience data fit with the overall approach, as well as also into a wider dataset of reporting.
· Think of XM and XLAs as an evolution of service management and SLAs, not a completely new idea.
10. Review and constantly improve the model.
· Start to use the data outputs to develop and improve services.
· Keep working on improving the mix of metrics and relative weightings
· Regularly validate and iterate with customers and users.
· Build up some data on successes achieved and publicise – service improvement, Feedback scores, cost, quality and efficiency improvements, innovations.
· Keep doing all of this with CSI.
Challenges
There are several key challenges as mentioned above:
- Failure to learn from SLA mistakes
- Missing key user input and requirements
- Over-engineering the process – surveys and data
- Managing disinterest from customers and businesses
- Setting off without support and understanding of the value expected
- Availability of skilled resources
The other major challenge is also around how to make the ‘agreement’ part of this work. As mentioned, the value of this work is found mostly in the process of developing it.
However, the format of a successful XLA agreement should reflect the spirit and culture of this, rather than becoming a burden. Simple clear documentation should be used. This is still a new and mostly uncharted area.
How should the industry take this forward?
It should be clear from this blog that the development and use of XM and XLAs fits as part of Service Management evolution. This should not need to be seen as a separate or competing approach.
XLAs still require the core foundation of data and service definition that are found in SLAs and as part of ITSM. XLAs are an addition to SLAs, not a replacement for them.
As an industry we should also recognise this and work together, to develop an integrated approach to learning, implementation, tooling and our definitions of success.
This blog supports my conference presentation session at the UCISA SSG conference in Birmingham, UK on 4th July 2024. SSG24 – UCISA
“Experience management and AI are two main hype areas at present – these can work well together and add significant value to service delivery. The concept of ‘XLAs’ – experience level agreements – is also being heavily socialised. The core elements of XLAs are not new, although the context and approach are re-designed and re-packaged with new content, training courses and other developments, including now an ‘XLA Institute’.
The UCISA session provides clear direction and practical examples of how to make this work, based on the presenter’s experience, using a similar XLA approach with clients for over 10 years.
