SlideShare a Scribd company logo
1 of 111
Download to read offline
From Bean-Counter to Data Psychologist
Using Analytics to
Understand theWHY
Senior Partner, Analytics Demystified
Michele Kiss @michelejkiss
@michelejkiss
About Me
I’m originally Australian (not that you can tell anymore)
In my ‘free time’ I teach Les Mills Group Fitness classes (like BodyPump, CXWORX, RPM)
I have an unnatural obsession with my pets – a 9-year-old cat, Bella, and a dorky labradoodle
named Fenway
@michelejkiss
Analytics Demystified
I am a Senior Partner at Analytics Demystified. If you have been in this industry for even a relatively
short amount a time, it’s likely you have heard of Eric Peterson, John Lovett, Adam Greco (etc.)
@michelejkiss
We literally wrote the book...s
on digital analytics
When I say that Demystified WROTE the book on digital analytics, that’s not hyperbole. We literally
did.
@michelejkiss
So today we’re going to talk about what we sometimes forget about, in our highly quantified world:
the WHY. What lies behind the numbers?
Typical Analytics
Think of a typical analytics report.
Typical Analytics
“What happened?”
The information included tells us WHAT happened. Site traffic went up or down, these traffic sources
shifted, conversion rate declined.
@michelejkiss
Typical Analytics
+8%
Click-Through Rate!
-18%
Click-Through Rate!
NoSignificant
Difference!
Whichvariationperformedbetter
Even if you take an arguably “more visual” analytics report than a standard “charts and tables”
report, the ultimate message doesn’t change.
Which variation performed better?
But not why.
@michelejkiss
Typical Analytics
Select Brand
Select Style
Select Features
Detailed Product Page
Exit Site
8
9
%
5
4
%
7
6
%
5
2
%
4
8
%
2
7
%
21%
Browse byTheme
7%
62%
13%
Browse by Idea
Where do
others go?
Where do
others go?
Gray arrows: “Eventual conversion”
(may involve pages in between
steps)
Yellow arrows: Immediate next-page
User behaviour indicates some may
be confused (e.g. going back) or not
finding what they’re looking for
(abandoning path)
LEGEND
6%
Home Page
Another Site Area
17%
Home Page
9%
Exit Site
5%
Select Brand
8%
Where do
others go?
Exited Site
3%
Saw Survey on Page
19%
Back to Select Style
(ThesevisitorsaregoingdowntheCategorypath)
From here:
68% go to Other Area
9% Exit site
9% go back to Select Brand
3
2
%
Even if you look at data in great detail – it’s still telling us WHAT happened? But not why?
More is needed to understand why people behaved the way they did
@michelejkiss
Typical Analytics
Select Brand
Select Style
4
8
%
2
7
%
Where do
others go?
5%
Select Brand
Take ONE behavior… users who select a brand, then select a style… then go back to selecting a
brand.
@michelejkiss
Leaves a sensible analyst thinking… but why?
@michelejkiss
Hmmm…
For example, a client of mine recently conducted an A/B test. This was a properly created and split
test of a Home Page hero. The call to action drives users down a particular path, and one variation is
found to have an impact.
@michelejkiss
Hmmm…
However, on a totally unrelated path, a significant result is also found.
If you trust the numbers at face value, you might assume that the test affected this conversion.
However, if you stop and assess this critically, you might wonder – how can a hero image shown
pages earlier, that contains NO links to this path, truly affect conversion to it?
In other words… you have the results, but you wonder… WHY?
That’s not to say
that this would be better
“My Mom says
blue is the
better choice.”
@michelejkiss
What are people doing?
Web Analytics
Question
Which performs better?
Why do our users do/don’t do “X”?
Social Analytics
Surveys
A/B/MV Testing
User Testing
Session Replay
Voice of Customer
To truly understand behaviour, we need to understand a number of questions, and this involves a
number of different methods.
@michelejkiss
Essentially, we need to play a little armchair psychologist.
The Plural of
Anecdote
is not
DataHowever, that doesn’t mean just substituting qualitative data
as somehow being more powerful than quantitative data.
Rather, we want to harness multiple methods for getting a
more complete picture.
Art
Science After all, analytics is understanding human behavior (and
we are complicated!)… that’s nothing if not an art AND
science combined.
@michelejkiss
The Power of Data
How pervasive is the problem?
Commentary
How do people feel about it?
There is immense power in being able to use quantitative data to understand the magnitude of a
problem, together with qualitative data to unearth the feelings and motivations that lie behind that
action.
Which is more persuasive?
Only18%progressthrough
Step1ofourSignupFlow
100%
18%
7%
6%
Step 1
Step 2
Step 3
Complete
100%
18%
7%
6%
Step 1
Step 2
Step 3
Complete
“Whytheareyouaskingmeformybirthdateand
streetaddresstodownloadawhitepaper?!”
-Actual customer feedback based on Step 1 of our form
These methods are
pervasive in other fields
These methods are pervasive in other fields
But we don’t use them nearly enough
@michelejkiss
The medical field – we normally think of as being “all quantified” – lab tests and MRIs.
@michelejkiss
Medical
EmpiricalTests Patient-Reported History
However, the medical field actually combines data from tests together with patient reported history
and symptoms. Diagnosis can’t be based purely on one or the other.
@michelejkiss
Medical
In a season three episode of House, they deal with a car-crash patient who has a disorder called CIPA, which is a rare
inherited disorder of the nervous system which prevents the sensation of pain, heat, and cold.
Life with CIPA means constantly checking your mouth to make sure you didn’t bite it, your teeth to ensure none of them
have fallen out or broken, your eyes to make sure you didn’t scratch a cornea, your limbs to make sure you haven’t
broken a bone… even setting an alarm just to remind you to go to the bathroom.
It also means when you go to the hospital and there’s something wrong, you are NO help to the diagnosis because you
literally can’t tell the doctor what hurts.
In this case, the absence of this more “qualitative” data is literally a deal-breaker, that makes quantitative analysis of test
results more akin to “finding a needle in a haystack”
@michelejkiss
Journalism
Statistics
Quotes
Journalists do this without missing a beat.
After all, an article that just cited a bunch of numbers at you (especially in prose form, where it’s hard
for people to process numbers!) wouldn’t sway anyone.
But when statistics, speaking to the prevalence of the issue in question, are combined with quotes
that bring the story “to life”, you’ve got a much more persuasive article.
@michelejkiss
Legal
“Iwanttotellyouastory.I'mgoingtoaskyoualltocloseyoureyeswhileI
tellyouthestory.
Thisisastoryaboutalittlegirlwalkinghomefromthegrocerystoreone
sunnyafternoon.Iwantyoutopicturethislittlegirl.Suddenlyatruckraces
up.Twomenjumpoutandgrabher.Theydragherintoanearbyfieldand
theytieherupandtheyripherclothesfromherbody.Nowtheyclimb
on….
Canyouseeher?Herraped,beaten,brokenbody.Canyouseeher?Iwant
youtopicturethatlittlegirl.Nowimagineshe'swhite.”
Evidence
Argument
The legal field too ties hard evidence together with narrative, via the opening and closing arguments
of a trial.
@michelejkiss
Marketing
TestimonialsCustomer Numbers
Marketing does this constantly – using customer numbers and observed impacts on business to
persuade, but also testimonials, quotes and case studies.
@michelejkiss
Health
“Feel”
Measures
Even in health and fitness. There are a lot of ways you can track your progress to a fitness goal. For
example, losing weight, or measuring muscle quality. But you can also just go by “feel” – “my clothes
feel looser” (even if you haven’t lost a pound!)
So let’s say you’re sold.
How can you better sell your findings?
@michelejkiss
Surveys
@michelejkiss
Surveys
RedesignControl
A/BTest
Winner
But why?
@michelejkiss
Surveys
RedesignControl
A/BTest Survey
“Whatareyouhopingtofindonthiswebsite?”
Other Pricing Product Information Support
Capture the test variation for the survey results to compare across groups.Run surveys in tandem with A/B tests. (Even something as simple as “what were you hoping to find”
might, when you compare variations, help you directionally understand the difference between the
groups.)
Make sure you’re actually capturing which test version ties to the survey responses! You could do
that by passing the test ID to your survey tool, by passing the survey ID to your testing tool, or by
integrating the two in your web analytics tool.
@michelejkiss
Exit Surveys
You probably have to have been living under a rock to have never seen an exit survey…
@michelejkiss
Post-Conversion Surveys
@michelejkiss
For customers who did convert, what pushed them over the edge?
@michelejkiss
Fall Out Surveys
Why’dyouleaveme?!
On the flip side…
@michelejkiss
Task Success Surveys
@michelejkiss
Monitor Satisfaction Scores
“NetPromoterScore”,“WordofMouthIndex”andsimilar
@michelejkiss
User-Initiated Feedback
Tools like Opinion Lab make it possible for users to give feedback when THEY choose, rather than
irritating users with a pop-up
You can guess all day at
whybut it’s easier to just ask…
@michelejkiss
What Should I Ask?
Avinash’s“Three Questions”
! What was the purpose of your visit?
! Did you accomplish what you came for?
! (If not)Why not?
@michelejkiss
What Should I Ask?
! Would you recommend us to a friend?
@michelejkiss
What Should I Ask?
! Would you recommend us to a friend?
! How would you rate your visit today?
@michelejkiss
What Should I Ask?
! Would you recommend us to a friend?
! How would you rate your visit today?
! Did anything frustrate you on your visit
today? (If so, please explain)
@michelejkiss
What Should I Ask?
! Would you recommend us to a friend?
! How would you rate your visit today?
! Did anything frustrate you on your visit
today? (If so, please explain)
! What did you like best/least about your
visit today?
@michelejkiss
Keep In Mind
No one is willing to fill this in:
@michelejkiss
Keep In Mind
Your goal is to get qualitative information
(Leantowardsfree-form,open-endedquestions)
Yes, it’s way easier to “count” and present questions where customers simply picked “Yes” or “No” or
rated on a scale of 1-10, but here you’re actually looking for their own voice, since this is to
supplement the wealth of quantitative data that you already have.
@michelejkiss
UserTesting
@michelejkiss
UserTesting
Remote UserTestingFormal UserTesting
Observation
Focus Groups
Eye tracking
Task completion
Card sorting
@michelejkiss
Focus Groups
Researcher gathers a group of people and asks questions and uses their discussions and interactions to
better understand the user perspective on the research question.
Funnily enough, I was part of a focus group back when Adobe first acquired Omniture.
It lasted about two hours, and they asked us a lot of questions about the perceptions of the two
companies. I also very clearly recall them asking for our opinion on various ways to rename the product.
And being miffed that they didn’t accept by “none of the above”!
E.g. They had “Adobe SiteCatalyst, powered by Omniture” or “SiteCatalyst, powered by Adobe” etc.
I wanted “Omniture SiteCatalyst, powered by Adobe.” ϑ
The researcher will normally try to find people in the right demographic for what they are looking for. For
example, in my case, they were looking for Omniture users.
Keep in mind… a focus group shouldn’t be your “source of truth” – it’s NEVER going to a representative
sample! Not only that but what people are willing to share in a focus group is not necessarily going to be
their honest truth. It’s an artificial environment and that must be kept in mind. (That’s why it’s good to use
focus groups to complement quantitative data but not assume they can stand 100% on their own.)
@michelejkiss
Observation &Task Completion
This is probably what you typically think of when you think of user testing –
Watching users actually interact with your product.
Observation can be done in-person, via recording or fully remotely.
This is often used with specific task completion exercises (rather than having the user just muddle about)
Task completion studies give the user specific things to do. These can be guided or unguided, and good
researchers will structure it so that (for example) the order of the tasks is randomized across participants,
to avoid fatigue, primary/recency effects, etc.
If you want to get a good example of how these go, check out … (next slide)
@michelejkiss
WantTo See It In Action?
Usertesting.com has a sample video that will show a remote, task-based user test
Really helpful to see if you’ve never gotten to observe user testing before
@michelejkiss
ObservationTesting Example
Example: Tire company.
Viewing a tire gave you an option to “Add”… which took you to something that looked like a cart, but
instead, was a quote.
Just analytics alone would not have revealed that users were confused as to why they couldn’t
purchase, and didn’t actually realize that they had to print off the quote and take it in to the store!
@michelejkiss
ObservationTesting Example
Allowed them to change the call to action to Get Quote, and put more clarification on the quote itself
that it was a quote only, and “these are the next steps to actually purchase these tires.”
@michelejkiss
Card Sorting
Card sorting exercises are often used to help understand how users “group” different pieces of
content or information, and can be useful to help inform Information Architecture
For example, what things do users think should be under a “Products & Services” menu? What
should be under the “About Us” menu?
The card sorting can be open or closed.
Open: Users can bucket the items in to any categories they want
Closed: Users are given the categories and need to assign each item
@michelejkiss
PrototypeTesting
One of the advantages of user testing over (or in conjunction with) A/B testing is that you can test concepts before
they have been fully developed.
This might include testing early working versions, but it could be as basic as doing task completion studies on
wireframes or sketches.
I recently participated in this with a company called Whistle, who make a doggie activity monitor. I did an exercise
where they had printed screenshots of different screens of their new app experience. I was assigned a task, and on
each screen I was asked a question about what I would do next, in order to complete the task.
Doing this quickly revealed to them that some of the CTAs they had named weren’t intuitive to the user (me.) And
what’s even cooler is that the new version of the app just updated on my phone, so I can see it actually come to life!
@michelejkiss
PrototypeTesting Example
+ SurveyA client of mine wanted to create a non-branded lifestyle app. They recruited a group of beta testers to use the app for
four weeks, and combined Google Analytics tracking of in-app behaviour with surveys throughout the beta. Together
this allowed them to understand what people actually did (especially vs. what they SAID they did!), how they used the
app, whether they would continue to use it and also how it affected their perception of the brand.
The data coupled was incredibly powerful – features that users claimed were important to them were in fact not used.
This allowed them to conduct follow up research to understand whether this was just a case of people not being
honest about what mattered, or whether it was poor app design and users couldn’t find the features!
@michelejkiss
So How Can I
You already have a job. You’re an analyst. You’re not going to go out there and become a fully
dedicated UX expert.
However, you should be thinking about how you can get insight from these types of research.
@michelejkiss
Sit In On UserTests
One option:
If your organization has a user experience group that runs regular user tests, take the time to sit in
on them from time to time.
You can do so in the room, or often remotely.
@michelejkiss
RunYour Own
You don’t have to “take over” user testing in your organization
There’s no reason that the analytics team can’t use user testing to inform test or analysis ideas
@michelejkiss
No Funding?
@michelejkiss
DIY UserTesting
Ask your family and friends for help! Ask them to perform behaviours on your site or app, and record
or take notes. You can easily use software like Camtasia to record their on-screen actions for
analysis later.
If you have a user experience team, ask for their help to properly structure tasks and questions.
You could also have
a little fun with it…
@michelejkiss
#justsayin
@michelejkiss
Don’t Forget…Think ofYour Audience
User testing isn’t about getting a bunch of random people off the street.
Think about WHO you are talking to and hearing from.
If you care about how to improve loyalty, it’s worth talking to your loyal customers.
If you want to improve the first-time experience, you’ll want a totally different segment.
Analyzing the right segment is just as important here as segmentation is in analytics.
@michelejkiss
Session Replay
Lots of vendors (Decibel Insights, ClickTale, Foreesee, Tealeaf and more) offer the ability to replay
user’s sessions.
This data is also typically aggregated in to heatmaps (since watching tons of sessions can be time
consuming!)
While you won’t get the commentary that typically accompanies user testing, it can be enlightening
to see what users actually DO when they interact with your site.
@michelejkiss
Customer Service Data
Most companies have SOME kind of customer feedback process, whether it be notes from phone calls, a general
comment email address, or a comment form.
Consider reaching out to your customer service team to see how you can work together to get access to this
information.
Keep in mind – you should be working with these people already! Often times they bear the wrath of angry customers
when (for example) A/B tests are not functioning properly. So you want them to be in the loop on what’s going on your
site or app, and likewise they can share valuable information with you about what customers are saying!
@michelejkiss
Social Listening
Another option to get qualitative feedback is to leverage social listening.
Even if this isn’t a formal program, with a proper listening tool, it’s not hard to review comments on
your brand’s Facebook page, or read through mentions on Twitter.
@michelejkiss
Personas
Claire
•  Health-conscious working
mom, a runner and cyclist
•  Wants to make nutritional
choices for her family
•  Loves using technology to
make her life easier
Another way you can tie quantitative and qualitative is to work with your user experience team (or
agency) to create analytics segments based on those personas.
Obviously not every item in the persona will be possible, however use the data you have available as
best you can.
@michelejkiss
Personas
Persona
•  Health-conscious working
mom, a runner and cyclist
•  Wants to make nutritional
choices for her family
•  Loves using technology to
make her life easier
Online
•  Online hours: Post-8PM
(after kids are in bed)
•  Frequent visits to
nutrition & recipe content
•  Logs in to save recipe, or
sends to email
Brainstorm what behaviours you think “Claire” would perform on your website or in your app, based
on her persona
Validate those with your UX team and ideally, with the “Claire’s” that your UX team talks to (e.g. what
content on your site interests them? Make sure it aligns with what you have included in your
segment)
This is also a great way of interesting your UX team in your web analytics data – when they can view
how people behave by their personas, they’re liking to better understand the data
@michelejkiss
Be your user
You’d be surprised how many analysts take shortcuts in actually experiencing the user experience.
For example, troubleshooting your mobile form…. From a desktop computer with User Agent
switching in place to mimic a mobile device.
You can’t truly gauge the user experience unless you do so in the same way as your user!
@michelejkiss
So now, theWHEN
@michelejkiss
When
ing
an analysisan analysis
Qualitative data can be used to help inform analysis ideas. For example, something you might not
have even realized is an issue for your users might come out from customer service feedback, or
from on-site surveys or user testing. Analysis will then help you quantify how big a problem it is, and
testing will allow you to optimize to improve the experience.
@michelejkiss
While running
tests
Running surveys alongside you’re A/B Tests can shed valuable light on why certain variations might
perform differently.
User testing (even while live!) of your A/B versions can add commentary to why people are behaving
the way they are.
@michelejkiss
When looking for opportunities
Tell me if this sounds familiar: You’ve been running an optimization program for a few years. You’ve
already tested all the easy stuff. And you’re tapped out for ideas.
Why not let your users decide what you’re missing?
@michelejkiss
During a redesign
User feedback is critical when you are trying to redesign an experience.
While you might have specific ideas about what you want to do, qualitative feedback can provide
critical insight in to alternate approaches that might actually work better with your users.
You can use this at all stages of the process – before any ideas are even thrown around, while
you’re designing (especially user testing to try out the concepts you’re discussing) and up until you
launch. (Then, hopefully at the same time as you A/B test your new designs against the existing
site!)
@michelejkiss
Pssssst…
Youaren’tlimitedtojust
testingyourownsite!
@michelejkiss
Test AgainstYour Competition
You can user-test any public site. Why not test the same feature on your site vs. your competitor’s?
(For example.)
@michelejkiss
What can we do?
1.Work with other teams
If you have teams dedicated to gathering qualitative insights (e.g. user experience teams, customer
service or consumer feedback, survey or research) start by trying to work with them, to leverage
what they’re already doing. (You don’t need to reinvent the wheel within your department…) Besides,
while their work will inform yours, your work can also help give insight in to theirs.
@michelejkiss
Bring the teams together
If you are in a manager or in a position to control location, co-locating your analytics and qualitative
teams (like user experience or consumer feedback) can help build cooperation and sharing amongst
the two teams.
@michelejkiss
Encourage a dialogue
For example, at Demystified we use Slack for internal communication. Using a discussion tool like
Slack can be valuable to help different departments discuss and debate ideas.
@michelejkiss
Share Findings
Working on an analysis? Just wrapped up something insightful that might be of use? Meet with the
team to walk them through it, and see if they have any qualitative information that would lend greater
insight to what you are working on.
@michelejkiss
Allow the time…
While adding qualitative insights to analytics work can be critical to better understanding how your
users behave, if your analytics team simply don’t have time, this helps no one. (It will just make them
feel like there is one more thing they’re failing at!) Managers need to account for the additional effort
by allowing additional time for results and analysis, and prioritizing appropriately.
@michelejkiss
Share Access
Often, companies have access to many different tools. However, access isn’t available to everyone
(and even if it is, people don’t necessarily have the training they would need to use them.)
If you are going to have your analysts leveraging additional data sources, it’s critical they have easy
access to the information, the necessary training to feel comfortable using the tools, and people to
ask questions to.
@michelejkiss
Where Possible, Integrate Data
1.70% 1.72%
1.85%
Detractors: 0-6 Passives: 7-8 Promoters: 9-10
Conversion Rate by Net Promoter Score
Many survey tools offer the ability to integrate their result data in to your web analytics data.
This allows you to segment and understand on-site behavior in light of survey responses.
For example, does Net Promoter Score correlate to conversion rate?
Do people who said they could NOT complete their task tend to fill in your contact form more or less?
We found this extremely useful in a previous company, where we actually ran surveys on behalf of advertisers.
While the survey was itself revenue-generating, we obviously didn’t want the survey to hurt overall revenue, since that could lead to a bigger hit for the business.
Even at its most basic, we were able to examine the exit rates of users who were presented with, and completed, the survey, to see if this was sending users OFF the site.
(It turns out, it was.) This was then used to put strict rules in place about the frequency and targeting of surveys, to allow the overall survey product to be revenue positive by
limiting the impact to other businesses.
2. Invest
If you don’t have teams or tools in place to start leveraging more qualitative data for your analysis, you’ll need to budget for it.
This doesn’t have to be piles of money…
You can conduct inexpensive user testing online for minimal investment, and there are plenty of free survey solutions out there.
Consider using a tag manager to allow the team to easily deploy surveys. (GTM and Adobe DTM are both free!!)
There ARE free options you can leverage. And use the results you get from those to sell the need to invest more in this area.
@michelejkiss
What can I do?
So what if it’s just you?
You don’t have a manager who is willing to spear-head this effort, but you know it could add real
value to the work you do?
@michelejkiss
You’ll have to go it alone…
Free solutions will be your saving grace: Free surveys, free tag managers.
Run ad-hoc user tests, even if it’s only on your family and friends.
Use trial versions of technologies to prove out the value for a particular analysis.
@michelejkiss
Don’t be above bribes!
If there ARE people in your organization who can help, you just don’t have the official “clout” to
demand they help, don’t overlook the power of bribery. Taking someone to lunch and getting their
thoughts or help, or bribing another team with treats, can be surprisingly successful.
@michelejkiss
Your plan of attack
So, what can you do now?
@michelejkiss
Your plan of attack
! Run a survey alongside your next A/B test
@michelejkiss
Your plan of attack
! Run a survey alongside your next A/B test
! Do a simple‘user test’(family or friend!) on
your next analysis topic
@michelejkiss
Your plan of attack
! Run a survey alongside your next A/B test
! Do a simple‘user test’(family or friend!) on
your next analysis topic
! Make friends with UX + Customer Service
But wait…
How do I present this stuff?
Use
“Quotes”
“Ididnotfindthesoftware,and
didnotwanttobrowsethrough
hundredsofundescribed
options”
-Actual customer feedback
after failing to find software
Only 11%of users
who search for downloads
successfully
download software
@michelejkiss
Share themes
Word Clouds can be a useful way of just giving the gist of the overall feedback, without users
needing to actually read everything or without having to do complicated sentiment analysis. Critical
themes and words will stand out.
While these don’t replace concrete data, they can add insight and help “round out” the picture.
@michelejkiss
Chart Feedback
Positive
Negative
$10
Plan
$5
Plan
Free
Plan
You can chart feedback e.g. on two axes.
How positive vs. negative was each piece of feedback and, let’s say you know a second variable
(like what plan the customer was on) adding that on the second axis so you have a quadrant.
@michelejkiss
Chart Feedback
Positive
Negative
$10
Plan
$5
Plan
Free
Plan
This will allow you to focus on the positive, high-value feedback
@michelejkiss
Chart Feedback
Positive
Negative
$10
Plan
$5
Plan
Free
Plan
Or the negative, low-value feedback
@michelejkiss
Group Feedback
Read About Email Product
Read About the company
Checkout Pricing
See Admin Options
Get Support
Get Multi-User Support
Learn About Cloud Sync
Learn AboutYour Security
Preview Calendar App
Learn About Phone
Other
What is the main purpose of your visit?
Grouping feedback in to categories can make it more easy to digest, and easier to overlay with web
analytics data to draw comparisons between what people said they came for, and what they actually
did.
Keep in mind…
Qualitative data is not
better or worse than quantitative.
It’s just different.
And gives us a different picture.
The
Hawthorne
Effect
After all, there’s an entire theory around how our behaviour changes when we are observed, referred
to as the Hawthorne effect. Certain research methods like user testing might be more affected by this
than, say, web analytics is. For this reason, combining what someone says they are interested in
with how they actually behaved can be immensely valuable, as their behaviour might have changed
just by being asked.
@michelejkiss
The moral of the story: The more data you can use to understand the bigger picture, and not be
limited just by one tool, the better your analysis will be at uncovering the motives behind your user’s
actions – and the better your marketing can be at addressing those motives and concerns.
Michele Kiss
Senior Partner
Analytics Demystified

More Related Content

What's hot

"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change
"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change
"What?", "So what?", "NOW WHAT?" How to influence people and accomplish changeLarry Maccherone
 
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...Troy Magennis
 
Content Jam 2014 - James Ellis - Analytics Without Fear
Content Jam 2014 - James Ellis - Analytics Without FearContent Jam 2014 - James Ellis - Analytics Without Fear
Content Jam 2014 - James Ellis - Analytics Without FearOrbit Media Studios
 
5 Marketing Automation Tips for Small Businesses
5 Marketing Automation Tips for Small Businesses 5 Marketing Automation Tips for Small Businesses
5 Marketing Automation Tips for Small Businesses Mathew Sweezey
 
Growth Hackers of Vienna - Meetup #2 Part1&2
Growth Hackers of Vienna - Meetup #2  Part1&2Growth Hackers of Vienna - Meetup #2  Part1&2
Growth Hackers of Vienna - Meetup #2 Part1&2Growth Hackers of Vienna
 
Digital Analytics Checkup: How to evaluate the impact of your web analytics data
Digital Analytics Checkup: How to evaluate the impact of your web analytics dataDigital Analytics Checkup: How to evaluate the impact of your web analytics data
Digital Analytics Checkup: How to evaluate the impact of your web analytics dataCrossView
 
10 steps to online fundraising and fundraising analytics
10 steps to online fundraising and fundraising analytics10 steps to online fundraising and fundraising analytics
10 steps to online fundraising and fundraising analyticsThe Safdar Group
 
[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...
[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...
[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...CXL
 
How to Increase Your Testing Success by Combining Qualitative and Quantitativ...
How to Increase Your Testing Success by Combining Qualitative and Quantitativ...How to Increase Your Testing Success by Combining Qualitative and Quantitativ...
How to Increase Your Testing Success by Combining Qualitative and Quantitativ...Optimizely
 
Hits: A look at web metrics
Hits: A look at web metricsHits: A look at web metrics
Hits: A look at web metricsMitchell Hislop
 
Are You Finding True Root Causes?
Are You Finding True Root Causes?Are You Finding True Root Causes?
Are You Finding True Root Causes?EMMAIntl
 
[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...
[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...
[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...CXL
 
[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...
[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...
[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...CXL
 
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in researchThe researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in researchRuth Ellison
 
Froomle Tech Webinar : Stand on the Giants
Froomle Tech Webinar : Stand on the GiantsFroomle Tech Webinar : Stand on the Giants
Froomle Tech Webinar : Stand on the GiantsFroomle
 
Digital morning, Stockholm, 23 of March
Digital morning, Stockholm, 23 of MarchDigital morning, Stockholm, 23 of March
Digital morning, Stockholm, 23 of MarchEva-Lotta Laurin
 
Validation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based  product management by Abdallah Al-KhalidiValidation and hypothesis based  product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-KhalidiAbdallah Al-Khalidi
 
Interpretations of data
Interpretations of dataInterpretations of data
Interpretations of dataBusiness901
 

What's hot (20)

"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change
"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change
"What?", "So what?", "NOW WHAT?" How to influence people and accomplish change
 
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...
Risk Management and Reliable Forecasting using Un-reliable Data (magennis) - ...
 
Content Jam 2014 - James Ellis - Analytics Without Fear
Content Jam 2014 - James Ellis - Analytics Without FearContent Jam 2014 - James Ellis - Analytics Without Fear
Content Jam 2014 - James Ellis - Analytics Without Fear
 
5 Marketing Automation Tips for Small Businesses
5 Marketing Automation Tips for Small Businesses 5 Marketing Automation Tips for Small Businesses
5 Marketing Automation Tips for Small Businesses
 
Growth Hackers of Vienna - Meetup #2 Part1&2
Growth Hackers of Vienna - Meetup #2  Part1&2Growth Hackers of Vienna - Meetup #2  Part1&2
Growth Hackers of Vienna - Meetup #2 Part1&2
 
Digital Analytics Checkup: How to evaluate the impact of your web analytics data
Digital Analytics Checkup: How to evaluate the impact of your web analytics dataDigital Analytics Checkup: How to evaluate the impact of your web analytics data
Digital Analytics Checkup: How to evaluate the impact of your web analytics data
 
10 steps to online fundraising and fundraising analytics
10 steps to online fundraising and fundraising analytics10 steps to online fundraising and fundraising analytics
10 steps to online fundraising and fundraising analytics
 
[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...
[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...
[CXL Live 16] From WTF to Hell Yes - How to Come Up With Copy That Persuades ...
 
How to Increase Your Testing Success by Combining Qualitative and Quantitativ...
How to Increase Your Testing Success by Combining Qualitative and Quantitativ...How to Increase Your Testing Success by Combining Qualitative and Quantitativ...
How to Increase Your Testing Success by Combining Qualitative and Quantitativ...
 
Hits: A look at web metrics
Hits: A look at web metricsHits: A look at web metrics
Hits: A look at web metrics
 
Are You Finding True Root Causes?
Are You Finding True Root Causes?Are You Finding True Root Causes?
Are You Finding True Root Causes?
 
[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...
[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...
[CXL Live 16] Neuro-Persuasion - Brain-Based Strategies for Online Marketers ...
 
HOW TO THINK LIKE A DATA SCIENTIST
HOW TO THINK LIKE A DATA SCIENTISTHOW TO THINK LIKE A DATA SCIENTIST
HOW TO THINK LIKE A DATA SCIENTIST
 
[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...
[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...
[Elite Camp 2016] Amy Harrison - The Science of Screenwriting for Online Mark...
 
The Data Game
The Data GameThe Data Game
The Data Game
 
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in researchThe researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research
 
Froomle Tech Webinar : Stand on the Giants
Froomle Tech Webinar : Stand on the GiantsFroomle Tech Webinar : Stand on the Giants
Froomle Tech Webinar : Stand on the Giants
 
Digital morning, Stockholm, 23 of March
Digital morning, Stockholm, 23 of MarchDigital morning, Stockholm, 23 of March
Digital morning, Stockholm, 23 of March
 
Validation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based  product management by Abdallah Al-KhalidiValidation and hypothesis based  product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-Khalidi
 
Interpretations of data
Interpretations of dataInterpretations of data
Interpretations of data
 

Viewers also liked

Accelerate 2011-super-accelerator-tim wilson-final-post
Accelerate 2011-super-accelerator-tim wilson-final-postAccelerate 2011-super-accelerator-tim wilson-final-post
Accelerate 2011-super-accelerator-tim wilson-final-postTim Wilson
 
Gilligan's Guide to Analysts as Community Managers' Best Friends
Gilligan's Guide to Analysts as Community Managers' Best FriendsGilligan's Guide to Analysts as Community Managers' Best Friends
Gilligan's Guide to Analysts as Community Managers' Best FriendsTim Wilson
 
Bizwatch Search Analytics Solving PPC & SEO Problemswith Bizwatch
Bizwatch Search Analytics Solving PPC & SEO Problemswith BizwatchBizwatch Search Analytics Solving PPC & SEO Problemswith Bizwatch
Bizwatch Search Analytics Solving PPC & SEO Problemswith BizwatchLaura Thieme
 
Top 10 Things I Wish I Knew When I Started In Web Analytics
Top 10 Things I Wish I Knew When I Started In Web AnalyticsTop 10 Things I Wish I Knew When I Started In Web Analytics
Top 10 Things I Wish I Knew When I Started In Web AnalyticsJune Dershewitz
 
Facebook Measurement Fundamentals
Facebook Measurement FundamentalsFacebook Measurement Fundamentals
Facebook Measurement FundamentalsTim Wilson
 
Data Visualization: Cognitive Psychology and Design Principles
Data Visualization:  Cognitive Psychology and Design PrinciplesData Visualization:  Cognitive Psychology and Design Principles
Data Visualization: Cognitive Psychology and Design PrinciplesDan Sweet
 
Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010
Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010
Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010CRM Metrix - MetrixLab
 
A Process for Being Data Driven
A Process for Being Data DrivenA Process for Being Data Driven
A Process for Being Data DrivenTim Wilson
 
Damion Brown - The Missing Automation Layer - Superweek 2017
Damion Brown - The Missing Automation Layer - Superweek 2017Damion Brown - The Missing Automation Layer - Superweek 2017
Damion Brown - The Missing Automation Layer - Superweek 2017Damion Brown
 
Should Digital Analysts Become More Data Science-y?
Should Digital Analysts Become More Data Science-y?Should Digital Analysts Become More Data Science-y?
Should Digital Analysts Become More Data Science-y?Tim Wilson
 
Social Marketing Analytics: A New Framework for Measuring Results in Social M...
Social Marketing Analytics: A New Framework for Measuring Results in Social M...Social Marketing Analytics: A New Framework for Measuring Results in Social M...
Social Marketing Analytics: A New Framework for Measuring Results in Social M...John Lovett
 
Data Visualization Tips and Concepts
Data Visualization Tips and ConceptsData Visualization Tips and Concepts
Data Visualization Tips and ConceptsTim Wilson
 
The Science of Timing
The Science of TimingThe Science of Timing
The Science of TimingHubSpot
 
Web Analytics Tools Comparison
Web Analytics Tools ComparisonWeb Analytics Tools Comparison
Web Analytics Tools ComparisonTim Wilson
 
Reinforcement learning conductrics-superweek2017
Reinforcement learning conductrics-superweek2017Reinforcement learning conductrics-superweek2017
Reinforcement learning conductrics-superweek2017Matt Gershoff
 
Sample slides by Garr Reynolds
Sample slides by Garr ReynoldsSample slides by Garr Reynolds
Sample slides by Garr Reynoldsgarr
 

Viewers also liked (18)

Accelerate 2011-super-accelerator-tim wilson-final-post
Accelerate 2011-super-accelerator-tim wilson-final-postAccelerate 2011-super-accelerator-tim wilson-final-post
Accelerate 2011-super-accelerator-tim wilson-final-post
 
Gilligan's Guide to Analysts as Community Managers' Best Friends
Gilligan's Guide to Analysts as Community Managers' Best FriendsGilligan's Guide to Analysts as Community Managers' Best Friends
Gilligan's Guide to Analysts as Community Managers' Best Friends
 
Facebook Measurement
Facebook MeasurementFacebook Measurement
Facebook Measurement
 
Bizwatch Search Analytics Solving PPC & SEO Problemswith Bizwatch
Bizwatch Search Analytics Solving PPC & SEO Problemswith BizwatchBizwatch Search Analytics Solving PPC & SEO Problemswith Bizwatch
Bizwatch Search Analytics Solving PPC & SEO Problemswith Bizwatch
 
Top 10 Things I Wish I Knew When I Started In Web Analytics
Top 10 Things I Wish I Knew When I Started In Web AnalyticsTop 10 Things I Wish I Knew When I Started In Web Analytics
Top 10 Things I Wish I Knew When I Started In Web Analytics
 
Facebook Measurement Fundamentals
Facebook Measurement FundamentalsFacebook Measurement Fundamentals
Facebook Measurement Fundamentals
 
Game On: Gaming Mechanics
Game On: Gaming Mechanics Game On: Gaming Mechanics
Game On: Gaming Mechanics
 
Data Visualization: Cognitive Psychology and Design Principles
Data Visualization:  Cognitive Psychology and Design PrinciplesData Visualization:  Cognitive Psychology and Design Principles
Data Visualization: Cognitive Psychology and Design Principles
 
Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010
Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010
Hemen Patel CRM Metrix: Columbus WAW, Jan 27, 2010
 
A Process for Being Data Driven
A Process for Being Data DrivenA Process for Being Data Driven
A Process for Being Data Driven
 
Damion Brown - The Missing Automation Layer - Superweek 2017
Damion Brown - The Missing Automation Layer - Superweek 2017Damion Brown - The Missing Automation Layer - Superweek 2017
Damion Brown - The Missing Automation Layer - Superweek 2017
 
Should Digital Analysts Become More Data Science-y?
Should Digital Analysts Become More Data Science-y?Should Digital Analysts Become More Data Science-y?
Should Digital Analysts Become More Data Science-y?
 
Social Marketing Analytics: A New Framework for Measuring Results in Social M...
Social Marketing Analytics: A New Framework for Measuring Results in Social M...Social Marketing Analytics: A New Framework for Measuring Results in Social M...
Social Marketing Analytics: A New Framework for Measuring Results in Social M...
 
Data Visualization Tips and Concepts
Data Visualization Tips and ConceptsData Visualization Tips and Concepts
Data Visualization Tips and Concepts
 
The Science of Timing
The Science of TimingThe Science of Timing
The Science of Timing
 
Web Analytics Tools Comparison
Web Analytics Tools ComparisonWeb Analytics Tools Comparison
Web Analytics Tools Comparison
 
Reinforcement learning conductrics-superweek2017
Reinforcement learning conductrics-superweek2017Reinforcement learning conductrics-superweek2017
Reinforcement learning conductrics-superweek2017
 
Sample slides by Garr Reynolds
Sample slides by Garr ReynoldsSample slides by Garr Reynolds
Sample slides by Garr Reynolds
 

Similar to Using Analytics and Surveys to Understand User Behavior

Slides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassLean Analytics
 
AI Fails: Avoiding bias in your systems
AI Fails: Avoiding bias in your systemsAI Fails: Avoiding bias in your systems
AI Fails: Avoiding bias in your systemsDr Janet Bastiman
 
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?Kongregate
 
Digital analytics: Wrap-up (Lecture 12)
Digital analytics: Wrap-up (Lecture 12)Digital analytics: Wrap-up (Lecture 12)
Digital analytics: Wrap-up (Lecture 12)Joni Salminen
 
Online Course: Real Statistics: A Radical Approach
Online Course: Real Statistics: A Radical ApproachOnline Course: Real Statistics: A Radical Approach
Online Course: Real Statistics: A Radical ApproachAsad Zaman
 
Book Summary : Everybody Lies
Book Summary : Everybody LiesBook Summary : Everybody Lies
Book Summary : Everybody LiesRahul Rishi
 
Data is love data viz best practices
Data is love   data viz best practicesData is love   data viz best practices
Data is love data viz best practicesGregory Nelson
 
Social Media Research: with focus on Twitter and misinformation
Social Media Research: with focus on Twitter and misinformationSocial Media Research: with focus on Twitter and misinformation
Social Media Research: with focus on Twitter and misinformationkhornberger
 
online startups metrics
online startups metrics online startups metrics
online startups metrics Hatem Kameli
 
A predictive analytics primer
A predictive analytics primerA predictive analytics primer
A predictive analytics primerShesha
 
Module 8 discussion.docx
Module 8 discussion.docxModule 8 discussion.docx
Module 8 discussion.docxstudywriters
 
Diamonds in the Rough (Sentiment(al) Analysis
Diamonds in the Rough (Sentiment(al) AnalysisDiamonds in the Rough (Sentiment(al) Analysis
Diamonds in the Rough (Sentiment(al) AnalysisScott K. Wilder
 
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docxBUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docxjasoninnes20
 
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docxBUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docxcurwenmichaela
 
Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics
 
'Drinking from the fire hose? The pitfalls and potential of Big Data'.
'Drinking from the fire hose? The pitfalls and potential of Big Data'.'Drinking from the fire hose? The pitfalls and potential of Big Data'.
'Drinking from the fire hose? The pitfalls and potential of Big Data'.Josh Cowls
 

Similar to Using Analytics and Surveys to Understand User Behavior (20)

Slides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclass
 
AI Fails: Avoiding bias in your systems
AI Fails: Avoiding bias in your systemsAI Fails: Avoiding bias in your systems
AI Fails: Avoiding bias in your systems
 
Jerait PDF.pdf
Jerait PDF.pdfJerait PDF.pdf
Jerait PDF.pdf
 
Being a Data-Driven Communicator
Being a Data-Driven CommunicatorBeing a Data-Driven Communicator
Being a Data-Driven Communicator
 
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
Emily Greer at GDC 2018: Data-Driven or Data-Blinded?
 
Digital analytics: Wrap-up (Lecture 12)
Digital analytics: Wrap-up (Lecture 12)Digital analytics: Wrap-up (Lecture 12)
Digital analytics: Wrap-up (Lecture 12)
 
Practical Social Analytics
Practical Social AnalyticsPractical Social Analytics
Practical Social Analytics
 
Online Course: Real Statistics: A Radical Approach
Online Course: Real Statistics: A Radical ApproachOnline Course: Real Statistics: A Radical Approach
Online Course: Real Statistics: A Radical Approach
 
Book Summary : Everybody Lies
Book Summary : Everybody LiesBook Summary : Everybody Lies
Book Summary : Everybody Lies
 
Data is love data viz best practices
Data is love   data viz best practicesData is love   data viz best practices
Data is love data viz best practices
 
Social Media Research: with focus on Twitter and misinformation
Social Media Research: with focus on Twitter and misinformationSocial Media Research: with focus on Twitter and misinformation
Social Media Research: with focus on Twitter and misinformation
 
online startups metrics
online startups metrics online startups metrics
online startups metrics
 
A predictive analytics primer
A predictive analytics primerA predictive analytics primer
A predictive analytics primer
 
Module 8 discussion.docx
Module 8 discussion.docxModule 8 discussion.docx
Module 8 discussion.docx
 
Unit 2.pptx
Unit 2.pptxUnit 2.pptx
Unit 2.pptx
 
Diamonds in the Rough (Sentiment(al) Analysis
Diamonds in the Rough (Sentiment(al) AnalysisDiamonds in the Rough (Sentiment(al) Analysis
Diamonds in the Rough (Sentiment(al) Analysis
 
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docxBUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
 
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docxBUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
BUS 308 Week 2 Lecture 1 Examining Differences - overview .docx
 
Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and Enterprises
 
'Drinking from the fire hose? The pitfalls and potential of Big Data'.
'Drinking from the fire hose? The pitfalls and potential of Big Data'.'Drinking from the fire hose? The pitfalls and potential of Big Data'.
'Drinking from the fire hose? The pitfalls and potential of Big Data'.
 

Recently uploaded

Agencia Marketing Branding Measurement Certification Google Ads Abril 2024
Agencia Marketing Branding Measurement Certification Google Ads Abril 2024Agencia Marketing Branding Measurement Certification Google Ads Abril 2024
Agencia Marketing Branding Measurement Certification Google Ads Abril 2024Marketing BRANDING
 
The power of SEO-driven market intelligence
The power of SEO-driven market intelligenceThe power of SEO-driven market intelligence
The power of SEO-driven market intelligenceHinde Lamrani
 
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local LeadsSearch Engine Journal
 
TAM Sports IPL 17 Advertising Report- M01 - M23
TAM Sports IPL 17 Advertising Report- M01 - M23TAM Sports IPL 17 Advertising Report- M01 - M23
TAM Sports IPL 17 Advertising Report- M01 - M23Social Samosa
 
Creating a Successful Digital Marketing Campaign.pdf
Creating a Successful Digital Marketing Campaign.pdfCreating a Successful Digital Marketing Campaign.pdf
Creating a Successful Digital Marketing Campaign.pdfgopzzzin
 
Digital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G AgeDigital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G AgeDigiKarishma
 
Professional Sales Representative by Sahil Srivastava.pptx
Professional Sales Representative by Sahil Srivastava.pptxProfessional Sales Representative by Sahil Srivastava.pptx
Professional Sales Representative by Sahil Srivastava.pptxSahil Srivastava
 
5 Digital Marketing Tips | Devherds Software Solutions
5 Digital Marketing Tips | Devherds Software Solutions5 Digital Marketing Tips | Devherds Software Solutions
5 Digital Marketing Tips | Devherds Software SolutionsDevherds Software Solutions
 
Miss Immigrant USA Activity Pageant Program.pdf
Miss Immigrant USA Activity Pageant Program.pdfMiss Immigrant USA Activity Pageant Program.pdf
Miss Immigrant USA Activity Pageant Program.pdfMagdalena Kulisz
 
The Evolution of Internet : How consumers use technology and its impact on th...
The Evolution of Internet : How consumers use technology and its impact on th...The Evolution of Internet : How consumers use technology and its impact on th...
The Evolution of Internet : How consumers use technology and its impact on th...sowmyrao14
 
Navigating Global Markets and Strategies for Success
Navigating Global Markets and Strategies for SuccessNavigating Global Markets and Strategies for Success
Navigating Global Markets and Strategies for SuccessElizabeth Moore
 
15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing Strategy15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing StrategyBlue Atlas Marketing
 
Introduction to marketing Management Notes
Introduction to marketing Management NotesIntroduction to marketing Management Notes
Introduction to marketing Management NotesKiranTiwari42
 
Ryanair Marketing and business development Plan
Ryanair Marketing and business development  PlanRyanair Marketing and business development  Plan
Ryanair Marketing and business development Plandrkarimsaber
 
Understand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMMUnderstand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMMsearchextensionin
 
Digital Marketing complete introduction.
Digital Marketing complete introduction.Digital Marketing complete introduction.
Digital Marketing complete introduction.Kashish Bindra
 
Content Marketing: How To Find The True Value Of Your Marketing Funnel
Content Marketing: How To Find The True Value Of Your Marketing FunnelContent Marketing: How To Find The True Value Of Your Marketing Funnel
Content Marketing: How To Find The True Value Of Your Marketing FunnelSearch Engine Journal
 
Master the art of Social Selling to increase sales by fostering relationships...
Master the art of Social Selling to increase sales by fostering relationships...Master the art of Social Selling to increase sales by fostering relationships...
Master the art of Social Selling to increase sales by fostering relationships...VereigenMedia1
 
20 Top Social Media Tips for Peer Specialists
20 Top Social Media Tips for Peer Specialists20 Top Social Media Tips for Peer Specialists
20 Top Social Media Tips for Peer Specialistsmlicam615
 
Paul Russell Confidential Resume for Fahlo.pdf
Paul Russell Confidential Resume for Fahlo.pdfPaul Russell Confidential Resume for Fahlo.pdf
Paul Russell Confidential Resume for Fahlo.pdfpaul8402
 

Recently uploaded (20)

Agencia Marketing Branding Measurement Certification Google Ads Abril 2024
Agencia Marketing Branding Measurement Certification Google Ads Abril 2024Agencia Marketing Branding Measurement Certification Google Ads Abril 2024
Agencia Marketing Branding Measurement Certification Google Ads Abril 2024
 
The power of SEO-driven market intelligence
The power of SEO-driven market intelligenceThe power of SEO-driven market intelligence
The power of SEO-driven market intelligence
 
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
 
TAM Sports IPL 17 Advertising Report- M01 - M23
TAM Sports IPL 17 Advertising Report- M01 - M23TAM Sports IPL 17 Advertising Report- M01 - M23
TAM Sports IPL 17 Advertising Report- M01 - M23
 
Creating a Successful Digital Marketing Campaign.pdf
Creating a Successful Digital Marketing Campaign.pdfCreating a Successful Digital Marketing Campaign.pdf
Creating a Successful Digital Marketing Campaign.pdf
 
Digital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G AgeDigital Marketing in 5G Era - Digital Transformation in 5G Age
Digital Marketing in 5G Era - Digital Transformation in 5G Age
 
Professional Sales Representative by Sahil Srivastava.pptx
Professional Sales Representative by Sahil Srivastava.pptxProfessional Sales Representative by Sahil Srivastava.pptx
Professional Sales Representative by Sahil Srivastava.pptx
 
5 Digital Marketing Tips | Devherds Software Solutions
5 Digital Marketing Tips | Devherds Software Solutions5 Digital Marketing Tips | Devherds Software Solutions
5 Digital Marketing Tips | Devherds Software Solutions
 
Miss Immigrant USA Activity Pageant Program.pdf
Miss Immigrant USA Activity Pageant Program.pdfMiss Immigrant USA Activity Pageant Program.pdf
Miss Immigrant USA Activity Pageant Program.pdf
 
The Evolution of Internet : How consumers use technology and its impact on th...
The Evolution of Internet : How consumers use technology and its impact on th...The Evolution of Internet : How consumers use technology and its impact on th...
The Evolution of Internet : How consumers use technology and its impact on th...
 
Navigating Global Markets and Strategies for Success
Navigating Global Markets and Strategies for SuccessNavigating Global Markets and Strategies for Success
Navigating Global Markets and Strategies for Success
 
15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing Strategy15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing Strategy
 
Introduction to marketing Management Notes
Introduction to marketing Management NotesIntroduction to marketing Management Notes
Introduction to marketing Management Notes
 
Ryanair Marketing and business development Plan
Ryanair Marketing and business development  PlanRyanair Marketing and business development  Plan
Ryanair Marketing and business development Plan
 
Understand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMMUnderstand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMM
 
Digital Marketing complete introduction.
Digital Marketing complete introduction.Digital Marketing complete introduction.
Digital Marketing complete introduction.
 
Content Marketing: How To Find The True Value Of Your Marketing Funnel
Content Marketing: How To Find The True Value Of Your Marketing FunnelContent Marketing: How To Find The True Value Of Your Marketing Funnel
Content Marketing: How To Find The True Value Of Your Marketing Funnel
 
Master the art of Social Selling to increase sales by fostering relationships...
Master the art of Social Selling to increase sales by fostering relationships...Master the art of Social Selling to increase sales by fostering relationships...
Master the art of Social Selling to increase sales by fostering relationships...
 
20 Top Social Media Tips for Peer Specialists
20 Top Social Media Tips for Peer Specialists20 Top Social Media Tips for Peer Specialists
20 Top Social Media Tips for Peer Specialists
 
Paul Russell Confidential Resume for Fahlo.pdf
Paul Russell Confidential Resume for Fahlo.pdfPaul Russell Confidential Resume for Fahlo.pdf
Paul Russell Confidential Resume for Fahlo.pdf
 

Using Analytics and Surveys to Understand User Behavior

  • 1. From Bean-Counter to Data Psychologist Using Analytics to Understand theWHY Senior Partner, Analytics Demystified Michele Kiss @michelejkiss
  • 2. @michelejkiss About Me I’m originally Australian (not that you can tell anymore) In my ‘free time’ I teach Les Mills Group Fitness classes (like BodyPump, CXWORX, RPM) I have an unnatural obsession with my pets – a 9-year-old cat, Bella, and a dorky labradoodle named Fenway
  • 3. @michelejkiss Analytics Demystified I am a Senior Partner at Analytics Demystified. If you have been in this industry for even a relatively short amount a time, it’s likely you have heard of Eric Peterson, John Lovett, Adam Greco (etc.)
  • 4. @michelejkiss We literally wrote the book...s on digital analytics When I say that Demystified WROTE the book on digital analytics, that’s not hyperbole. We literally did.
  • 5. @michelejkiss So today we’re going to talk about what we sometimes forget about, in our highly quantified world: the WHY. What lies behind the numbers?
  • 6. Typical Analytics Think of a typical analytics report.
  • 7. Typical Analytics “What happened?” The information included tells us WHAT happened. Site traffic went up or down, these traffic sources shifted, conversion rate declined.
  • 8. @michelejkiss Typical Analytics +8% Click-Through Rate! -18% Click-Through Rate! NoSignificant Difference! Whichvariationperformedbetter Even if you take an arguably “more visual” analytics report than a standard “charts and tables” report, the ultimate message doesn’t change. Which variation performed better? But not why.
  • 9. @michelejkiss Typical Analytics Select Brand Select Style Select Features Detailed Product Page Exit Site 8 9 % 5 4 % 7 6 % 5 2 % 4 8 % 2 7 % 21% Browse byTheme 7% 62% 13% Browse by Idea Where do others go? Where do others go? Gray arrows: “Eventual conversion” (may involve pages in between steps) Yellow arrows: Immediate next-page User behaviour indicates some may be confused (e.g. going back) or not finding what they’re looking for (abandoning path) LEGEND 6% Home Page Another Site Area 17% Home Page 9% Exit Site 5% Select Brand 8% Where do others go? Exited Site 3% Saw Survey on Page 19% Back to Select Style (ThesevisitorsaregoingdowntheCategorypath) From here: 68% go to Other Area 9% Exit site 9% go back to Select Brand 3 2 % Even if you look at data in great detail – it’s still telling us WHAT happened? But not why? More is needed to understand why people behaved the way they did
  • 10. @michelejkiss Typical Analytics Select Brand Select Style 4 8 % 2 7 % Where do others go? 5% Select Brand Take ONE behavior… users who select a brand, then select a style… then go back to selecting a brand.
  • 11. @michelejkiss Leaves a sensible analyst thinking… but why?
  • 12. @michelejkiss Hmmm… For example, a client of mine recently conducted an A/B test. This was a properly created and split test of a Home Page hero. The call to action drives users down a particular path, and one variation is found to have an impact.
  • 13. @michelejkiss Hmmm… However, on a totally unrelated path, a significant result is also found. If you trust the numbers at face value, you might assume that the test affected this conversion. However, if you stop and assess this critically, you might wonder – how can a hero image shown pages earlier, that contains NO links to this path, truly affect conversion to it? In other words… you have the results, but you wonder… WHY?
  • 14. That’s not to say that this would be better
  • 15. “My Mom says blue is the better choice.”
  • 16. @michelejkiss What are people doing? Web Analytics Question Which performs better? Why do our users do/don’t do “X”? Social Analytics Surveys A/B/MV Testing User Testing Session Replay Voice of Customer To truly understand behaviour, we need to understand a number of questions, and this involves a number of different methods.
  • 17. @michelejkiss Essentially, we need to play a little armchair psychologist.
  • 18. The Plural of Anecdote is not DataHowever, that doesn’t mean just substituting qualitative data as somehow being more powerful than quantitative data. Rather, we want to harness multiple methods for getting a more complete picture.
  • 19. Art Science After all, analytics is understanding human behavior (and we are complicated!)… that’s nothing if not an art AND science combined.
  • 20. @michelejkiss The Power of Data How pervasive is the problem? Commentary How do people feel about it? There is immense power in being able to use quantitative data to understand the magnitude of a problem, together with qualitative data to unearth the feelings and motivations that lie behind that action.
  • 21. Which is more persuasive?
  • 23. 100% 18% 7% 6% Step 1 Step 2 Step 3 Complete “Whytheareyouaskingmeformybirthdateand streetaddresstodownloadawhitepaper?!” -Actual customer feedback based on Step 1 of our form
  • 24. These methods are pervasive in other fields These methods are pervasive in other fields But we don’t use them nearly enough
  • 25. @michelejkiss The medical field – we normally think of as being “all quantified” – lab tests and MRIs.
  • 26. @michelejkiss Medical EmpiricalTests Patient-Reported History However, the medical field actually combines data from tests together with patient reported history and symptoms. Diagnosis can’t be based purely on one or the other.
  • 27. @michelejkiss Medical In a season three episode of House, they deal with a car-crash patient who has a disorder called CIPA, which is a rare inherited disorder of the nervous system which prevents the sensation of pain, heat, and cold. Life with CIPA means constantly checking your mouth to make sure you didn’t bite it, your teeth to ensure none of them have fallen out or broken, your eyes to make sure you didn’t scratch a cornea, your limbs to make sure you haven’t broken a bone… even setting an alarm just to remind you to go to the bathroom. It also means when you go to the hospital and there’s something wrong, you are NO help to the diagnosis because you literally can’t tell the doctor what hurts. In this case, the absence of this more “qualitative” data is literally a deal-breaker, that makes quantitative analysis of test results more akin to “finding a needle in a haystack”
  • 28. @michelejkiss Journalism Statistics Quotes Journalists do this without missing a beat. After all, an article that just cited a bunch of numbers at you (especially in prose form, where it’s hard for people to process numbers!) wouldn’t sway anyone. But when statistics, speaking to the prevalence of the issue in question, are combined with quotes that bring the story “to life”, you’ve got a much more persuasive article.
  • 30. @michelejkiss Marketing TestimonialsCustomer Numbers Marketing does this constantly – using customer numbers and observed impacts on business to persuade, but also testimonials, quotes and case studies.
  • 31. @michelejkiss Health “Feel” Measures Even in health and fitness. There are a lot of ways you can track your progress to a fitness goal. For example, losing weight, or measuring muscle quality. But you can also just go by “feel” – “my clothes feel looser” (even if you haven’t lost a pound!)
  • 32. So let’s say you’re sold.
  • 33. How can you better sell your findings?
  • 36. @michelejkiss Surveys RedesignControl A/BTest Survey “Whatareyouhopingtofindonthiswebsite?” Other Pricing Product Information Support Capture the test variation for the survey results to compare across groups.Run surveys in tandem with A/B tests. (Even something as simple as “what were you hoping to find” might, when you compare variations, help you directionally understand the difference between the groups.) Make sure you’re actually capturing which test version ties to the survey responses! You could do that by passing the test ID to your survey tool, by passing the survey ID to your testing tool, or by integrating the two in your web analytics tool.
  • 37. @michelejkiss Exit Surveys You probably have to have been living under a rock to have never seen an exit survey…
  • 38. @michelejkiss Post-Conversion Surveys @michelejkiss For customers who did convert, what pushed them over the edge?
  • 42. @michelejkiss User-Initiated Feedback Tools like Opinion Lab make it possible for users to give feedback when THEY choose, rather than irritating users with a pop-up
  • 43. You can guess all day at whybut it’s easier to just ask…
  • 44. @michelejkiss What Should I Ask? Avinash’s“Three Questions” ! What was the purpose of your visit? ! Did you accomplish what you came for? ! (If not)Why not?
  • 45. @michelejkiss What Should I Ask? ! Would you recommend us to a friend?
  • 46. @michelejkiss What Should I Ask? ! Would you recommend us to a friend? ! How would you rate your visit today?
  • 47. @michelejkiss What Should I Ask? ! Would you recommend us to a friend? ! How would you rate your visit today? ! Did anything frustrate you on your visit today? (If so, please explain)
  • 48. @michelejkiss What Should I Ask? ! Would you recommend us to a friend? ! How would you rate your visit today? ! Did anything frustrate you on your visit today? (If so, please explain) ! What did you like best/least about your visit today?
  • 49. @michelejkiss Keep In Mind No one is willing to fill this in:
  • 50. @michelejkiss Keep In Mind Your goal is to get qualitative information (Leantowardsfree-form,open-endedquestions) Yes, it’s way easier to “count” and present questions where customers simply picked “Yes” or “No” or rated on a scale of 1-10, but here you’re actually looking for their own voice, since this is to supplement the wealth of quantitative data that you already have.
  • 53. @michelejkiss Focus Groups Researcher gathers a group of people and asks questions and uses their discussions and interactions to better understand the user perspective on the research question. Funnily enough, I was part of a focus group back when Adobe first acquired Omniture. It lasted about two hours, and they asked us a lot of questions about the perceptions of the two companies. I also very clearly recall them asking for our opinion on various ways to rename the product. And being miffed that they didn’t accept by “none of the above”! E.g. They had “Adobe SiteCatalyst, powered by Omniture” or “SiteCatalyst, powered by Adobe” etc. I wanted “Omniture SiteCatalyst, powered by Adobe.” ϑ The researcher will normally try to find people in the right demographic for what they are looking for. For example, in my case, they were looking for Omniture users. Keep in mind… a focus group shouldn’t be your “source of truth” – it’s NEVER going to a representative sample! Not only that but what people are willing to share in a focus group is not necessarily going to be their honest truth. It’s an artificial environment and that must be kept in mind. (That’s why it’s good to use focus groups to complement quantitative data but not assume they can stand 100% on their own.)
  • 54. @michelejkiss Observation &Task Completion This is probably what you typically think of when you think of user testing – Watching users actually interact with your product. Observation can be done in-person, via recording or fully remotely. This is often used with specific task completion exercises (rather than having the user just muddle about) Task completion studies give the user specific things to do. These can be guided or unguided, and good researchers will structure it so that (for example) the order of the tasks is randomized across participants, to avoid fatigue, primary/recency effects, etc. If you want to get a good example of how these go, check out … (next slide)
  • 55. @michelejkiss WantTo See It In Action? Usertesting.com has a sample video that will show a remote, task-based user test Really helpful to see if you’ve never gotten to observe user testing before
  • 56. @michelejkiss ObservationTesting Example Example: Tire company. Viewing a tire gave you an option to “Add”… which took you to something that looked like a cart, but instead, was a quote. Just analytics alone would not have revealed that users were confused as to why they couldn’t purchase, and didn’t actually realize that they had to print off the quote and take it in to the store!
  • 57. @michelejkiss ObservationTesting Example Allowed them to change the call to action to Get Quote, and put more clarification on the quote itself that it was a quote only, and “these are the next steps to actually purchase these tires.”
  • 58. @michelejkiss Card Sorting Card sorting exercises are often used to help understand how users “group” different pieces of content or information, and can be useful to help inform Information Architecture For example, what things do users think should be under a “Products & Services” menu? What should be under the “About Us” menu? The card sorting can be open or closed. Open: Users can bucket the items in to any categories they want Closed: Users are given the categories and need to assign each item
  • 59. @michelejkiss PrototypeTesting One of the advantages of user testing over (or in conjunction with) A/B testing is that you can test concepts before they have been fully developed. This might include testing early working versions, but it could be as basic as doing task completion studies on wireframes or sketches. I recently participated in this with a company called Whistle, who make a doggie activity monitor. I did an exercise where they had printed screenshots of different screens of their new app experience. I was assigned a task, and on each screen I was asked a question about what I would do next, in order to complete the task. Doing this quickly revealed to them that some of the CTAs they had named weren’t intuitive to the user (me.) And what’s even cooler is that the new version of the app just updated on my phone, so I can see it actually come to life!
  • 60. @michelejkiss PrototypeTesting Example + SurveyA client of mine wanted to create a non-branded lifestyle app. They recruited a group of beta testers to use the app for four weeks, and combined Google Analytics tracking of in-app behaviour with surveys throughout the beta. Together this allowed them to understand what people actually did (especially vs. what they SAID they did!), how they used the app, whether they would continue to use it and also how it affected their perception of the brand. The data coupled was incredibly powerful – features that users claimed were important to them were in fact not used. This allowed them to conduct follow up research to understand whether this was just a case of people not being honest about what mattered, or whether it was poor app design and users couldn’t find the features!
  • 61. @michelejkiss So How Can I You already have a job. You’re an analyst. You’re not going to go out there and become a fully dedicated UX expert. However, you should be thinking about how you can get insight from these types of research.
  • 62. @michelejkiss Sit In On UserTests One option: If your organization has a user experience group that runs regular user tests, take the time to sit in on them from time to time. You can do so in the room, or often remotely.
  • 63. @michelejkiss RunYour Own You don’t have to “take over” user testing in your organization There’s no reason that the analytics team can’t use user testing to inform test or analysis ideas
  • 65. @michelejkiss DIY UserTesting Ask your family and friends for help! Ask them to perform behaviours on your site or app, and record or take notes. You can easily use software like Camtasia to record their on-screen actions for analysis later. If you have a user experience team, ask for their help to properly structure tasks and questions.
  • 66. You could also have a little fun with it…
  • 69. @michelejkiss Don’t Forget…Think ofYour Audience User testing isn’t about getting a bunch of random people off the street. Think about WHO you are talking to and hearing from. If you care about how to improve loyalty, it’s worth talking to your loyal customers. If you want to improve the first-time experience, you’ll want a totally different segment. Analyzing the right segment is just as important here as segmentation is in analytics.
  • 70. @michelejkiss Session Replay Lots of vendors (Decibel Insights, ClickTale, Foreesee, Tealeaf and more) offer the ability to replay user’s sessions. This data is also typically aggregated in to heatmaps (since watching tons of sessions can be time consuming!) While you won’t get the commentary that typically accompanies user testing, it can be enlightening to see what users actually DO when they interact with your site.
  • 71. @michelejkiss Customer Service Data Most companies have SOME kind of customer feedback process, whether it be notes from phone calls, a general comment email address, or a comment form. Consider reaching out to your customer service team to see how you can work together to get access to this information. Keep in mind – you should be working with these people already! Often times they bear the wrath of angry customers when (for example) A/B tests are not functioning properly. So you want them to be in the loop on what’s going on your site or app, and likewise they can share valuable information with you about what customers are saying!
  • 72. @michelejkiss Social Listening Another option to get qualitative feedback is to leverage social listening. Even if this isn’t a formal program, with a proper listening tool, it’s not hard to review comments on your brand’s Facebook page, or read through mentions on Twitter.
  • 73. @michelejkiss Personas Claire •  Health-conscious working mom, a runner and cyclist •  Wants to make nutritional choices for her family •  Loves using technology to make her life easier Another way you can tie quantitative and qualitative is to work with your user experience team (or agency) to create analytics segments based on those personas. Obviously not every item in the persona will be possible, however use the data you have available as best you can.
  • 74. @michelejkiss Personas Persona •  Health-conscious working mom, a runner and cyclist •  Wants to make nutritional choices for her family •  Loves using technology to make her life easier Online •  Online hours: Post-8PM (after kids are in bed) •  Frequent visits to nutrition & recipe content •  Logs in to save recipe, or sends to email Brainstorm what behaviours you think “Claire” would perform on your website or in your app, based on her persona Validate those with your UX team and ideally, with the “Claire’s” that your UX team talks to (e.g. what content on your site interests them? Make sure it aligns with what you have included in your segment) This is also a great way of interesting your UX team in your web analytics data – when they can view how people behave by their personas, they’re liking to better understand the data
  • 75. @michelejkiss Be your user You’d be surprised how many analysts take shortcuts in actually experiencing the user experience. For example, troubleshooting your mobile form…. From a desktop computer with User Agent switching in place to mimic a mobile device. You can’t truly gauge the user experience unless you do so in the same way as your user!
  • 77. @michelejkiss When ing an analysisan analysis Qualitative data can be used to help inform analysis ideas. For example, something you might not have even realized is an issue for your users might come out from customer service feedback, or from on-site surveys or user testing. Analysis will then help you quantify how big a problem it is, and testing will allow you to optimize to improve the experience.
  • 78. @michelejkiss While running tests Running surveys alongside you’re A/B Tests can shed valuable light on why certain variations might perform differently. User testing (even while live!) of your A/B versions can add commentary to why people are behaving the way they are.
  • 79. @michelejkiss When looking for opportunities Tell me if this sounds familiar: You’ve been running an optimization program for a few years. You’ve already tested all the easy stuff. And you’re tapped out for ideas. Why not let your users decide what you’re missing?
  • 80. @michelejkiss During a redesign User feedback is critical when you are trying to redesign an experience. While you might have specific ideas about what you want to do, qualitative feedback can provide critical insight in to alternate approaches that might actually work better with your users. You can use this at all stages of the process – before any ideas are even thrown around, while you’re designing (especially user testing to try out the concepts you’re discussing) and up until you launch. (Then, hopefully at the same time as you A/B test your new designs against the existing site!)
  • 82. @michelejkiss Test AgainstYour Competition You can user-test any public site. Why not test the same feature on your site vs. your competitor’s? (For example.)
  • 84. 1.Work with other teams If you have teams dedicated to gathering qualitative insights (e.g. user experience teams, customer service or consumer feedback, survey or research) start by trying to work with them, to leverage what they’re already doing. (You don’t need to reinvent the wheel within your department…) Besides, while their work will inform yours, your work can also help give insight in to theirs.
  • 85. @michelejkiss Bring the teams together If you are in a manager or in a position to control location, co-locating your analytics and qualitative teams (like user experience or consumer feedback) can help build cooperation and sharing amongst the two teams.
  • 86. @michelejkiss Encourage a dialogue For example, at Demystified we use Slack for internal communication. Using a discussion tool like Slack can be valuable to help different departments discuss and debate ideas.
  • 87. @michelejkiss Share Findings Working on an analysis? Just wrapped up something insightful that might be of use? Meet with the team to walk them through it, and see if they have any qualitative information that would lend greater insight to what you are working on.
  • 88. @michelejkiss Allow the time… While adding qualitative insights to analytics work can be critical to better understanding how your users behave, if your analytics team simply don’t have time, this helps no one. (It will just make them feel like there is one more thing they’re failing at!) Managers need to account for the additional effort by allowing additional time for results and analysis, and prioritizing appropriately.
  • 89. @michelejkiss Share Access Often, companies have access to many different tools. However, access isn’t available to everyone (and even if it is, people don’t necessarily have the training they would need to use them.) If you are going to have your analysts leveraging additional data sources, it’s critical they have easy access to the information, the necessary training to feel comfortable using the tools, and people to ask questions to.
  • 90. @michelejkiss Where Possible, Integrate Data 1.70% 1.72% 1.85% Detractors: 0-6 Passives: 7-8 Promoters: 9-10 Conversion Rate by Net Promoter Score Many survey tools offer the ability to integrate their result data in to your web analytics data. This allows you to segment and understand on-site behavior in light of survey responses. For example, does Net Promoter Score correlate to conversion rate? Do people who said they could NOT complete their task tend to fill in your contact form more or less? We found this extremely useful in a previous company, where we actually ran surveys on behalf of advertisers. While the survey was itself revenue-generating, we obviously didn’t want the survey to hurt overall revenue, since that could lead to a bigger hit for the business. Even at its most basic, we were able to examine the exit rates of users who were presented with, and completed, the survey, to see if this was sending users OFF the site. (It turns out, it was.) This was then used to put strict rules in place about the frequency and targeting of surveys, to allow the overall survey product to be revenue positive by limiting the impact to other businesses.
  • 91. 2. Invest If you don’t have teams or tools in place to start leveraging more qualitative data for your analysis, you’ll need to budget for it. This doesn’t have to be piles of money… You can conduct inexpensive user testing online for minimal investment, and there are plenty of free survey solutions out there. Consider using a tag manager to allow the team to easily deploy surveys. (GTM and Adobe DTM are both free!!) There ARE free options you can leverage. And use the results you get from those to sell the need to invest more in this area.
  • 92. @michelejkiss What can I do? So what if it’s just you? You don’t have a manager who is willing to spear-head this effort, but you know it could add real value to the work you do?
  • 93. @michelejkiss You’ll have to go it alone… Free solutions will be your saving grace: Free surveys, free tag managers. Run ad-hoc user tests, even if it’s only on your family and friends. Use trial versions of technologies to prove out the value for a particular analysis.
  • 94. @michelejkiss Don’t be above bribes! If there ARE people in your organization who can help, you just don’t have the official “clout” to demand they help, don’t overlook the power of bribery. Taking someone to lunch and getting their thoughts or help, or bribing another team with treats, can be surprisingly successful.
  • 95. @michelejkiss Your plan of attack So, what can you do now?
  • 96. @michelejkiss Your plan of attack ! Run a survey alongside your next A/B test
  • 97. @michelejkiss Your plan of attack ! Run a survey alongside your next A/B test ! Do a simple‘user test’(family or friend!) on your next analysis topic
  • 98. @michelejkiss Your plan of attack ! Run a survey alongside your next A/B test ! Do a simple‘user test’(family or friend!) on your next analysis topic ! Make friends with UX + Customer Service
  • 99. But wait… How do I present this stuff?
  • 101. “Ididnotfindthesoftware,and didnotwanttobrowsethrough hundredsofundescribed options” -Actual customer feedback after failing to find software Only 11%of users who search for downloads successfully download software
  • 102. @michelejkiss Share themes Word Clouds can be a useful way of just giving the gist of the overall feedback, without users needing to actually read everything or without having to do complicated sentiment analysis. Critical themes and words will stand out. While these don’t replace concrete data, they can add insight and help “round out” the picture.
  • 103. @michelejkiss Chart Feedback Positive Negative $10 Plan $5 Plan Free Plan You can chart feedback e.g. on two axes. How positive vs. negative was each piece of feedback and, let’s say you know a second variable (like what plan the customer was on) adding that on the second axis so you have a quadrant.
  • 104. @michelejkiss Chart Feedback Positive Negative $10 Plan $5 Plan Free Plan This will allow you to focus on the positive, high-value feedback
  • 106. @michelejkiss Group Feedback Read About Email Product Read About the company Checkout Pricing See Admin Options Get Support Get Multi-User Support Learn About Cloud Sync Learn AboutYour Security Preview Calendar App Learn About Phone Other What is the main purpose of your visit? Grouping feedback in to categories can make it more easy to digest, and easier to overlay with web analytics data to draw comparisons between what people said they came for, and what they actually did.
  • 107. Keep in mind… Qualitative data is not better or worse than quantitative.
  • 108. It’s just different. And gives us a different picture.
  • 109. The Hawthorne Effect After all, there’s an entire theory around how our behaviour changes when we are observed, referred to as the Hawthorne effect. Certain research methods like user testing might be more affected by this than, say, web analytics is. For this reason, combining what someone says they are interested in with how they actually behaved can be immensely valuable, as their behaviour might have changed just by being asked.
  • 110. @michelejkiss The moral of the story: The more data you can use to understand the bigger picture, and not be limited just by one tool, the better your analysis will be at uncovering the motives behind your user’s actions – and the better your marketing can be at addressing those motives and concerns.