Survey analysis in a nutshell with Jeffrey Henning

Please tweet! @jhenning @LoveStats #mrx
Survey Analysis
in a Nutshell
(A Peanut Shell)
Presented by Jeffrey Henning
President at Researchscape International
Moderated by Annie Pettit
Chief Research Officer at Peanut Labs

Our Agenda
Thinking about
analysis once the
results have come in
is too late!
Before Analysis
Start looking at each
question in isolation,
in detail.
It’s better together!
Group some
questions together,
and contrast
questions, too.
By Each Question InterrelationshipsCBA

BEFORE ANALYSIS

Before You Analyze
Before Completing
the questionnaire
design, make sure you
are answering the big
question. And adding
the little questions
you’ll need.
BC1

What’s the new price?
Will they buy it?
Are they happy?
THEBIGQUESTION
What do we do next?

True Stories...
“What media market do they live in?”
“Maybe we can estimate it from their area code,
assuming they haven’t moved.”
“Can you crosstab by county economic tier?”
“I guess we can lookup their county from their zip code,
and the tier from their county...”
“Let’s analyze the results by the annual revenue of the companies
the respondents work for.”
“We didn’t ask that.”
THElittleQUESTIONS

Before You Analyze
Before Completing
the questionnaire
you’ll need.
BC
Before Counting
Each response, make
sure you want to
include each
response. Clean your
data!
BCE21

Before You Analyze
Before Completing
the questionnaire
you’ll need.
BC
Before Counting
Each response, make
sure you want to
include each
response. Clean your
data!
After Data is in
hand, think of the
big picture. Plan
your timeline
accordingly.
BCE AD321

Love the Data but Leave the Data Behind
Data
Price = $2.99
Analysis An Answer

BY EACH QUESTION

Please rate your overall satisfaction with each of the
following:
Sample Size: 206 (71% of Respondents)

“Top 2 Box” Sums Top 2 Values

Means are Mean
Median Corrects for Skew without Requiring Judgment Calls
$4.69
$0.99
$2.00
$99.00
Hedges and Shah BMC Bioinformatics 2003 4:31 doi:10.1186/1471-2105-4-31

Why do you install new apps on your mobile device?

Why do you install new apps on your
mobile device?
• "I install new apps when I need to be entertained or when I need a tool that I don't
currently have handy such as a calculator or ruler."
• "I always want to be the first to try great apps that I will want to keep. Many of the
new apps that I solve help me solve a problem and I can convey this to my friends,
which makes me look good.”
• "Most of the time it's for entertainment purposes. I like having several app options in
case I am bored. Other times, it's because I need a specific app for something (map,
flight finder, photo editor, etc.)"
• "To be able to do things more efficiently, or to try and discover new games, or to use
apps with more functions and a better interface."
• "I usually download apps that add functionality to my phone, such as a flashlight app
or a sensor reading app. I don't download many game apps, and I almost never
download an app developed for a single business, like a banking app.”

INTERRELATIONSHIPS

Cross-tabulate Questions
Traditional Cross-tabs Draw Eye Away from Important Cells

Use Cell Comparisons When Possible
86% of iPad Users vs. 69% of Android Phone Users
Bought App in Past Month or Less

Average Related Questions into Indices
Customer Experience Index, Price Comparison Index, Customer
Satisfaction Index, and a Customer Loyalty Index
CUSTOMER
EXPERIENCE
Effectiveness Enjoyability
Ease
PRICE
COMPARISONS
To Expectations To Quality
To Competitors
CUSTOMER
LOYALTY
Repurchase
Likelihood
Reluctance to
Switch
Recommend
Likelihood
CUSTOMER
SATISFACTION
Satisfaction Expectations
Comparison with Ideal
Source: Researchscape International

Indices Highly Correlate to Loyalty Behavior
Rank CESL Metric Correlation Shared Variance
1 Customer Loyalty Index 0.762 58%
2 CSAT/Loyalty Index 0.751 56%
3 Champion Model 0.749 56%
4 Likelihood to Repurchase 0.730 53%
5 Likelihood to Recommend 0.728 53%
6 Index of Indices 0.685 47%
7 CSAT 0.639 41%
8 Apostle Model 0.604 36%
9 TNS Loyalty Model 0.590 35%
10 Reluctance to Switch 0.585 34%
18 NPS 0.502 25%
22 Ease 0.402 16%

Apostle Model
Contrast Questions with Quadrant Analyses
• Jones and Sasser pioneered
their own loyalty segmentation
• For satisfaction scale, use CSAT
question or ACSI’s 3 questions
• For loyalty, use likelihood to
repurchase or a loyalty index

Beyond Quadrant Analysis Tells a Story
53eig.ht/1JkdRVi

“Stories are just data with a soul.”
Story
Synthesis
Analysis
Data
- Brene Brown, TEDx Houston, 2010

Summary
Thinking about
analysis once the
results have come in
is too late!
Before Analysis
Start looking at each
question in isolation,
in detail.
It’s better together!
Group some
questions together,
and contrast
questions.
By Each Question InterrelationshipsCBA

Q&A: Contact Information
Annie Pettit
Chief Research Officer
annie@peanutlabs.com
ca.linkedin.com/in/AnniePettit/
twitter.com/LoveStats
facebook.com/AnniePettit
Jonathan Cheriff
Director of Sales & Marketing
jonathan.cheriff@peanutlabs.com
Find PeanutLabs on
LinkedIn Facebook Twitter YouTube
DIY POLLINGDIY SUPPORT
Jeffrey Henning
President
jhenning@researchscape.com
www.researchscape.com
twitter.com/jhenning
DIY SAMPLE

Survey analysis in a nutshell with Jeffrey Henning

Recommended

Recommended

More Related Content

What's hot

What's hot (11)

Similar to Survey analysis in a nutshell with Jeffrey Henning

Similar to Survey analysis in a nutshell with Jeffrey Henning (20)

More from Annie Pettit, Research Methodologist

More from Annie Pettit, Research Methodologist (20)

Recently uploaded

Recently uploaded (20)

Survey analysis in a nutshell with Jeffrey Henning