1. I’M A CHIEF EXPERIENCE OFFICER
USER EXPERIENCE
SLA
xLA

2. 02 USER EXPERIENCE WWW.GIARTE.COM
USER EXPERIENCE
I’m a Chief
Experience Officer
Written by Marco Gianotten
In the past, innovations like the PC and mobile phone first made their
entrance on the business market. Only much later, following substantial price
decreases, they became commonplace among consumers. Now the reverse is
true: companies mainly follow trends that emerge in the world of consumer
technology. The consequence: end users place high demands on Corporate
IT and behave like business-consumers. Many CIOs realise that user experi-
ence (UX) will play a prominent role within IT performance management.
This chapter discusses the why and how.
We live in an economy that increasingly
revolves around perception: what customers
feel, think and experience is the driving force.
Social media, comparison sites and user
reviews feed the experience economy and
change the way companies and customers
interact, not only digitally but also physically.
In the retail world, the credo is increasingly
‘bricks and clicks’: strong growth in online
retail is regularly associated with investments
in the conversion of counters, shops, show-
rooms and bank branches into experience cen-
tres and flagship stores. The experience
economy might look like a fad, but the way in
which managers’ measure the performance of
their own organisations has already changed
significantly. At more and more companies,
customer service is no longer judged in terms
of efficiency (for example, the number of min-
utes per customer contact or costs per FTE)
but in terms of customer experience. Many
companies rely on the Net Promoter Score: the
extent to which customers recommend the
company or product to friends and colleagues.
IT has a major impact on customer
experience
User eXperience (UX) plays a prominent role in
the services and extends to the design, the
marketing and the management of customer
channels, such as websites and contact centres.
Services and products may in no way harm the
customer experience. Designing, testing and
improving the usability of websites is common-
place in the world of e-commerce. Companies
do their utmost to help customers as much as
they can during the orientation and sales pro-
cess and actively try to avoid ‘shopping cart
abandonment’. In e-commerce, there is a direct
relationship between usability and commercial
ratios, such as average revenue per order and
repeat visits. Mobilising positive customer

3. 03 USER EXPERIENCE WWW.GIARTE.COM
experience produces sales. The growing influ-
ence of IT on our daily lives as consumers
makes an ever increasing impression on our
user experience. If online banking is briefly
unavailable, this immediately causes negative
emotions: that are socially contagious. The
problems with ING’s Dutch internet banking on
3 April 2013 produced more than twenty-
thousand tweets (#ING) in just a few hours.
The employee is also a consumer
In the experience economy, consumers hate it
when companies do not do what they promise,
if helpdesks do not really help or are hard to
reach, or if the information provided is inade-
quate or appears to be incorrect. The experi-
ence economy also hurtles into the world of
business IT. The modern consumer is not only
found in the high streets or online stores, but
also within your own organisation. There they
are ‘dressed’ as employees and end users, or the
internal IT customers. As end users, they adopt
more and more consumer behaviour: IT sys-
tems should ‘just work properly’ and the IT
department ‘should be easy to do business
with’. End users hate it when they have to call
back because the problem is not completely
solved, because they have to keep on repeating
what the problem is, or because they are always
faced with people who are not empowered to
solve the problem.
What does the end user actually want?
End users expect user-friendly business appli-
cations and portals, a decisive service desk and
smart mobile apps that allow them to monitor
or control key processes. As organisations dis-
cover that working entails more than bricks,
brains and bytes (the key elements of the New
Way of Working, short NWoW), the expecta-
tions of IT are raised. With another way of
working for employees, the emphasis shifts to
behaviour: the conduct, the content of the
work, working together in multidisciplinary
teams and learning by sharing knowledge in the
workplace with co-workers. The performance
of IT is becoming increasingly important here:
it affects the personal productivity of employ-
ees. Users (and their managers) are therefore
becoming more critical and verbal: they con-
User experience is more than just
technology.
The technical performance of systems is
often different from what users actually
experience. There may be a considerable
difference between the absolute availa-
bility of core systems (usually far above
99 percent on the basis of 168 hours per
week) and the perception of the users.
What do they notice in practice of less
than 1 percent technical downtime?
Health insurer Agis measured how its IT
users experience availability. Their expe-
rience was much worse than one would
suspect from the technical availability.
Every minute of downtime was magnified,
especially during peak periods; systems’
slow responsiveness was seen by users as
a form of unavailability.
Poor usability of business applications
leads to longer learning curves, lower
labour productivity and dissatisfaction.
Usability sometimes also comes under
pressure with off-the-shelf software. For
example, Microsoft was harshly criticised
during the introduction of its software
package for Windows 8 because of the
high learning curve and an inconsistent
user experience: Windows 8 works diffe-
rently on a tablet than on a desktop.

4. 04 USER EXPERIENCE WWW.GIARTE.COM
sider properly working IT as a matter of
hygiene; problems and obstacles in its use have
now become dissatisfiers. If users complain
about the sluggishness of a core application
such as CRM on their tablets, the IT depart-
ment does not stand on strong ground if it
argues that ‘the application is performing prop-
erly according to monitoring in the data centre’.
There is no point relying on user experience if
the negative user experiences are downplayed
or dismissed with the argument that ‘the SLA
has been complied with’. In other words: the
most important step is to recognise that expe-
rience, and thus the perception of the end user,
is the truth.
What do IT organisations do?
IT organisations are until now used to measur-
ing and managing mainly on the basis of techni-
cal performance. With classical office
automation, that was hardly ever a problem: IT
organisations – or their service providers in the
case of outsourcing – had their own servers,
networks, workstations and applications. If an
application on a PC was too slow or hard to
access, it was fairly easy to figure out the tech-
nical causes because IT had control over the
entire chain. Nowadays more and more func-
tionality is virtualised: delivered to notebooks,
tablets and smartphones – purchased by
employees themselves – by means of web appli-
We are
the 99%The end-users
many CIOs
realise that user
experience will
play a prominent
role within
performance
management

5. 05 USER EXPERIENCE WWW.GIARTE.COM
cations or with mobile apps. With trends like
Bring Your Own Device (BYOD) and mobile
internet, IT loses its absolute control over,
for example, workplace hardware, operating
systems on devices and networks. As a result,
control over user experience also declines. But
even without this control over the technology
you have to be accountable for managing and
improving user experience.
Classic SLAs are for technocrats
The good news is that more and more CIOs
recognise the importance of user experience.
More and more IT departments are conducting
research into user satisfaction. Nevertheless,
customer satisfaction and user friendliness are
often the poor relation in IT. After years of
relying on ‘hard’ data, it is not easy to give
space to subjective indicators. IT departments
have difficulty initiating the appropriate
The new measure for satisfaction.
Who dares?
In its new outsourcing contracts, ABN AMRO
has decided to rely on four KPIs that improve
the cooperation between all parties in all the
multi-vendor outsourced IT service supply-
chains: aka as collaborative KPIs. Two of the
four KPIs are subjective in nature: business
satisfaction and project excellence. Business
satisfaction is the annual score that key deci-
sion makers award the IT. Project excellence
is the evaluation of completed projects by
business sponsors. The satisfaction is mea-
sured on three points: a) quality of the deliv-
ered functionality, b) lead time and c) value
for money.
A few years ago, DSM decided to use KPIs in
which the voice of the end user is paramount.
As a single point of contact, it makes no sense
to close an incident administratively (closing
a ticket) with the aim of reaching agreements
about the average resolution time of the
SLA, while the problem is not properly
resolved for the user. The opinion of the IT
user has become overriding: after the ticket
has been closed, he assesses whether he is
satisfied with the solution by means of an
online survey.
In an updated contract with KPN, ING has
decided to work with a ‘new style’ SLA: an
experience level agreement (XLA). XLAs are
characterised by the lack of penalties, the
focus on continuous improvement and a
closed loop. If an individual user has a bad
experience following a service moment (inci-
dent handling, delivery from service cata-
logue, answer to a question), he or she is
called back to resolve the complaint. Even if
the cause of the complaint lies with the solu-
tion groups of third parties, KPN declares
ownership of the entire chain for resolving
incidents. This closed circle should lead to a
culture of improvement (instead of a claim
culture) where the user experience (rather
than technical performance) occupies cen-
tre stage. As with ABN AMRO, ING also has a
collaborative KPI for all parties (thus includ-
ing ING itself): the satisfaction of employ-
ees about their workplace is expressed by
a so-called happiness ratio. This is the ratio
of users who rate their workplace with a 7
or more to those who award a 5 or less. The
value 6 is neutral. The ratio is increased by
one point each year.

6. 06 USER EXPERIENCE WWW.GIARTE.COM
improvement processes on the basis of results.
The methods used do not always provide any
useful control information; sometimes manag-
ers do not know which buttons they should
press. For example, little is known about which
leading indicators ultimately affect the lagging
indicator user satisfaction.
Focussing on user experience means that
Service Level Agreements (SLAs) must be
adapted to the new reality: not only managing
on the basis of technical control ratios, but
also on the basis of output. On the one hand,
this output affects the impact of IT on the
business and, on the other hand, in the percep-
tion of customers and users. In outsourcing,
new agreements – in the direction of XLAs,
eXperience Level Agreements – will also
have to be made.
User satisfaction 2.0: hard and soft
User experience is becoming increasingly
important with both the design and functional-
ity of management and the corresponding ser-
vice. The measurement of user satisfaction
about the workplace or business applications
will have to be richer and deeper in order to
achieve a targeted improvement of quality.
It is important to combine soft data (opinions;
answers to open and closed questions) with
hard data (from IT Service Management sys-
tems, such as the active directory, service desk
tooling and CMDB) in order to identify causes
of discontent. In this way, differences become
clear between, for example, locations, type of
users and type of workplaces. The combined
data ensures that you can now provide the
answers. For the different categories of inci-
dents, what is the critical time period after
which the satisfaction plummets? Is there a
connection with working days or periods within
one day? And with the type of users? In addi-
tion to measuring the user experience, it is also
important to have access to tooling with which
user transactions can be measured for each
business process by means of a (mobile) device
and divided into components, such as render
time of the device, response time of the infra-
structure and the end-to-end transaction time.
Sentiments and discussions: the
social part of UX
By analysing the open comments in satisfaction
research (with the aid of language technology),
the sentiments about corporate IT can be made
clearly visible and connections found within
Focussing on user
experience means that SLAs
must be adapted to the
new reality

7. 07 USER EXPERIENCE WWW.GIARTE.COM
Net Promoter Score
In recent years, the Net Promoter Score (NPS)
has successfully established itself in the row
of top KPIs that (top) managers work with.
The NPS indicates the extent to which custo-
mers are willing to recommend a company
or organisation. It is now ten years since the
NPS was conceived by Frederick Reichheld.
He saw the NPS as a solution for the problem
that a high level of customer satisfaction did
not appear to predict the loyalty of custo-
mers. A disadvantage of working with the Net
Promoter Score is that it is not exclusive; the
figure is the total of the percentage of fans
that awards a 9 or 10 minus the percentage of
detractors that awards a 6 or less. The mid-
dle group of passives is not included in the
NPS score. An NPS of -5 can consist of 45%
promoters, 10% passives and 50% detractors,
but can also consist of 6% promoters, 88%
passives and 11% detractors. These are sub-
stantially different results with an identical
final score. In addition, managers tend to opt
for ‘low hanging fruit’: problem areas where
you can relatively quickly and easily imple-
ment improvements in order to quickly realise
higher NPS scores. NPS also takes no account
of the effect of outliers: an average favourable
NPS score produces a distorted image if the
5% most dissatisfied customers furiously stir
things up on social media. In other words, the
way in which the NPS is used also says some-
thing about the management culture within
an organisation.
Furthermore, considerable doubts are
expressed among consumers about the value
of the NPS. Kumar et al (2008) published an
article in the Harvard Business Review about
the results of an analysis carried out among
9,900 customers of a telecom company. It
emerged from this that 81% of telecom cus-
tomers said they would recommend their
provider but only 30% of them actually did so.
No more than 8% of those who received the
recommendation eventually became a profi-
table customer of the telecom company.
The good news is that UX can
clearly be managed and that the
results will contribute to a
stronger IT-business alignment

8. 08 USER EXPERIENCE WWW.GIARTE.COM
multinationals too. The answer not only lies in
measuring and clever analysis; the physical dia-
logue with users is also important to encourage
empathy. More and more IT organisations use
customer panels to collect feedback. Rabobank
regularly organises IT user panels with a mod-
erator; IT professionals are present as listeners
and feedback (active listening) what they have
learned and would do differently in the services.
UX with Net Promoter Score:
­sense of non-sense for IT?
There are various methods and sources avail-
able for monitoring the customer experience.
Some CIOs have implemented the Net
Promoter Score to measure the extent to which
users would recommend their own IT organisa-
tion. The advantage of the NPS is that this met-
ric is part of the marketing and sales
management lingua franca. However, there are
also a number of disadvantages attached to
working with NPS scores. Managers often do
not know what they can do to improve the rec-
ommended score. Furthermore, the NPS score
can be misleading and the question is also what
IT end users can and should do with the rec-
ommendation question, since they do not have
to deal with a free market with multiple provid-
ers for their IT.
Measuring user experience with
Customer Effort Score
In addition to the Net Promoter Score, a lot of
attention has been paid to a new metric during
the past five years or so. This is, more than the
NPS, focused on the service that companies
provide to their customers: the Customer
Effort Score (CES). The CES indicates the
amount of personal effort made by a customer
to get something done by the company. The
CES is applicable to proposals, questions,
orders, returns and changing data. The CES is a
five-point scale with the result being a number
between 1 and 5; the lower the number, the less
the effort. The CES is perfectly suitable as a
customer experience metric for IT organisa-
tions and can also be used in combination with
user satisfaction.
Another method that is widely used to monitor
the user experience is recording the user’s
opinion at ‘moments of truth’ (such as after the
delivery of a service or the resolution of an
incident) and then enriching this with ticket
data (such as the start and stop time, where the
elapsed time is the actual resolution time) in
order to identify causes of dissatisfaction in the
service chain. The recording of these opinions
during ‘moments of truth’ – often by means of
automated research following a telephone call
or email exchange – has already been used for
some time in the case of services provided to
consumers.
What is the ambition of the ­­­­
I­T organisation?
A service entity that mainly looks at the tech-
nology or a team that successfully contributes
to the growth and future of the business?
Those who have an eye for the latter, must be
open to matters such as (end) user experience
and experience level agreements. The good
news is that UX can clearly be managed and
that the results will contribute to a stronger
IT-business alignment.
GIARTE
P.O. Box 890
1000 AW Amsterdam
The Netherlands
Telephone +31 (0)20 622 3444
info@giarte.com
www.giarte.com