Surveys have a lot of tradition and norms behind them. And a lot of templates. Many of these templates were written many years ago when formal language made a lot more sense. Today, people expect casual and friendly language everywhere including from brands and companies. This presentation shows what happens to data quality and survey results when real language, not Charles Dickens language, is used.

1. #CRC2014 @LoveStats
A Tale of Two Surveys
Presented by
Annie Pettit, Chief Research Officer
Peanut Labs is a
company

2. #CRC2014 @LoveStats
It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of light, it was the season of darkness, it was the spring of hope, it was the winter of despair, we had everything before us, we had nothing before us, we were all going direct to heaven, we were all going direct the other way—in short, the period was so far like the present period, that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only.

3. #CRC2014 @LoveStats
Based on any advertising, such as TV commercials, roadside billboards, banners on internet websites, or other similar forms of advertising, please list in the textbox below the products or types of products that have you yourself have seen publicly advertised between January 1, 2013 and December 31, 2013. (Please type your answer in the box below.)

4. #CRC2014 @LoveStats
Based on any advertising, such as TV commercials, roadside billboards, banners on internet websites, or other similar forms of advertising, please list in the textbox below the products or types of products that have you yourself have seen publicly advertised between January 1, 2013 and December 31, 2013. (Please type your answer in the box below.)
What products did you see advertised last year?
What products did you see advertised last week?

5. #CRC2014 @LoveStats
Survey method
•Test n=530
•Control n=530
•US Census Rep
•Non-branded computer and video gaming survey
•20 minute survey

6. #CRC2014 @LoveStats
Traditional
Humanized
What is your favorite video/computer game?
If you could only play one video game for the rest of your life, what game would you choose?
Have you ever lined up at a store so you could be among the first to purchase a new video/computer game?
Are you one of those people who lines up at stores to be the first to buy a new game?
How much do you agree or disagree with these statements about gaming?
Love 'em or hate 'em, what do you think about video games?
What is your opinion about each of the following television shows?
Awesome or terrible, how do you rate these TV shows?
Is there any other information you would like to share with us regarding your opinions and use of video/computer games?
Sharing is caring! Is there anything else you'd like to share about gaming?

7. #CRC2014 @LoveStats
Data Quality

8. #CRC2014 @LoveStats
Speeding
•Failed if the survey was completed in the slowest 5% of completion times
0
1
2
3
4
5
Traditional Survey (%)
Humanized Survey (%)
Speeding

9. #CRC2014 @LoveStats
Straightlining
•Failed if data demonstrated straightlining to a half positive, half negative 8 question grid
0
5
10
15
20
Traditional Survey (%)
Humanized Survey (%)
Straightlining

10. #CRC2014 @LoveStats
Open Ends
•Failed if less than 10 characters were provided in response to a request for “3 reasons why…”
0
5
10
Traditional Survey (%)
Humanized Survey (%)
Open Ends

11. #CRC2014 @LoveStats
Red Herring
•Failed if two fake computer games were selected
0
1
2
3
4
5
Traditional Survey (%)
Humanized Survey (%)
Red Herring

12. #CRC2014 @LoveStats
Survey Evaluation
1.0
2.0
3.0
4.0
5.0
Too long
Too complex
Fun to answer
Would
recommend to
my friends
Traditional Wording
Fun Wording
Agree
Disagree

13. #CRC2014 @LoveStats
Responder Opinions
Traditional Survey
Humanized Survey
It was short and nice:)
nice short survey not enough points though
I found this to be an interesting survey. Thank you.
I love surveys and quizzes, and this one was very interesting, because i like sharing things about myself.
the survey should be worth more points considering how long the survey was
very original method of conducting a survey!
NA
This was one of the Best surveys I have done!!!
No
loved it.. funny and quick
Good
"May the survey FORCE be with you." Was a nice hilariously awesome thing to see. I laughed a lot. o3o
This survey was enjoyable.
this is an awesome survey. I thoroughly enjoyed myself while taking it.
i dont know.
good job, sir! If i had a fedora, i'd tip it in your direction.
It was good survey
I laughed out loud the way some questions were phrased, I wanted to say this is the most refreshing survey ive taken I had a lot of fun

14. #CRC2014 @LoveStats
Survey Results

15. #CRC2014 @LoveStats
Purchase Intent
0%
10%
20%
30%
40%
Not at all
Somewhat
Very
Extremely
Raw - Traditional
Raw - Humanized

16. #CRC2014 @LoveStats
What is a fair price?
0%
10%
20%
30%
$0
$1 to $14
$15 to $19
$20 to $29
$30 to $39
$40 to $49
$50 or more
Raw - Traditional
Raw - Humanized

17. #CRC2014 @LoveStats
Rating Ads
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
V
Unlikely
S
Unlikely
Neutral
S Likely
V Likely
Traditional
Humanized
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
V
Unlikely
S
Unlikely
Neutral
S Likely
V Likely
Traditional
Humanized
0.0%
10.0%
20.0%
30.0%
40.0%
V
Unlikely
S
Unlikely
Neutral
S Likely
V Likely
Traditional
Humanized
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
V
Unlikely
S
Unlikely
Neutral
S Likely
V Likely
Traditional
Humanized
On the package
On the products
Before the game
On the clothes

18. #CRC2014 @LoveStats
Implementing the results

19. #CRC2014 @LoveStats
There is no perfect question
Are you the primary grocery shopper in your household?
Which of the following statements best describes your role in buying groceries for your household?
Are you responsible for most of the grocery shopping for your household?
Do you normally do most of the grocery shopping for your household?

20. #CRC2014 @LoveStats
•Agree Strongly
•Agree Somewhat
•Neutral
•Disagree Somewhat
•Disagree Strongly
•Love
•Like
•Neutral
•Dislike
•Hate
Ask for simpler wording

21. #CRC2014 @LoveStats
Which of the following do you enjoy?
•Biking
•Cooking
•Hiking
•Reading
Which of the following do you enjoy?
•Bike
•Cook
•Hiking
•Read
Don’t compromise on grammar
Let’s eat Grandma. Let’s eat, Grandma. Punctuation saves lives.

22. #CRC2014 @LoveStats
For each of the following descriptions of shopping behaviors, please indicate whether the description is highly characteristic, somewhat characteristic, slightly characteristic, or not characteristic at all of you when you visit a membership-only warehouse club store.
How descriptive are these characteristics when you visit a membership-only warehouse club store.
Ask for shorter questions

23. #CRC2014 @LoveStats
Ask for a bit of humour
Religion Politics Sex Violence Race
Pets Movies Neutral jokes Common slang

24. #CRC2014 @LoveStats
Answer options can be fun too
What we usually say:
What you can say:
Every day
Every day. I think I'm addicted!
More than 10 hours
More than 10 hours. You can't make me stop!
No one, you normally play by yourself
Me, myself, and I
Never
Never. I have better things to do!
Zero
Zero, zilch, nada!

25. #CRC2014 @LoveStats
Thank you for your participation
You’ve been a great help. Thanks so much!
Say please and thank you

26. #CRC2014 @LoveStats
Let the people speak
What we usually say:
What you can say:
Please describe 3 reasons why that is your favorite video/computer game.
Give me 3 good reasons why!
Is there any other information you would like to share with us regarding your opinions and use of video/computer games? (Optional)
Sharing is caring! Is there anything else you'd like to share about gaming? (Answer if you wish)
Compared to other surveys you have taken, what is your opinion about this survey? (Please select one in each row)
What did you think of this survey? (Please select one in each row)
This is the last page of questions. Is there any other information you would like to share about this survey? (Optional)
This is the end, my friend. Is there anything about this survey you'd care to share? (Answer if you wish)

27. #CRC2014 @LoveStats
9/23/2014
27
Thank you!
Annie Pettit
Chief Research Officer
Peanut Labs
annie@peanutlabs.com
@LoveStats on twitter
facebook.com/anniepettit
ca.linkedin.com/in/anniepettit/
jonathan.cheriff@peanutlabs.com
: )
180 Montgomery Street, Suite 1700 - San Francisco, CA 94104