SlideShare ist ein Scribd-Unternehmen logo
1 von 27
Downloaden Sie, um offline zu lesen
Effects of Splitting 
Long Surveys Into Two 
Annie Pettit 
Chief Research Officer 
Tweet! #AAPOR @LoveStats
90 77 79 73 72 62 
43 40 34 31 21 14 
36 28 25 21 15 9 
http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/ 
Tweet! #AAPOR @LoveStats
50% of surveys are >=15 minutes 
5 10 15 20 25 30 35 40 45 50 55 60+ 
Tweet! #AAPOR @LoveStats 
Source: DIY Sampling Panel
Tweet! #AAPOR @LoveStats 
12 Minute 
Survey 
N=300 
Split 
incentive 
Method 
12 Minute 
Survey 
N=300 
Split 
incentive 
20 Minute 
Survey 
N=600 
Full 
incentive
Problem: 
Nothing is directly Comparable 
• People choose whether to participate 
• Each version is different 
– Different length 
– Different topics 
– Different leading/priming questions 
Tweet! #AAPOR @LoveStats
Tweet! #AAPOR @LoveStats 
Agenda 
1. Data quality 
2. Data equivalence 
3. Responder satisfaction 
11/6/2014 6
Data Quality 
11/6/2014 7 
Tweet! #AAPOR @LoveStats
Fewer Red Herrings 
    
Tweet! #AAPOR @LoveStats 
2 fake hotels 
2 fake airlines 
2 fake loyalty programs
Straightlining 
  
P<.05 
Tweet! #AAPOR @LoveStats 
50% 
40% 
30% 
20% 
10% 
0% 
Fail 
Straight 
1 
Fail 
Straight 
6 
Fail 
Straight 
10 
Fail 
Straight 
64 
Fail 
Straight 
66 
Fail 
Straight 
85 
Fail 
Straight 
91 
Fail 
Straight 
93 
Fail 
Straight 
104 
Long Short A Short B
Follow Instructions Better 
  
Did not follow instruction to “choose only one” 
Tweet! #AAPOR @LoveStats
Better Open Ends 
     
Tweet! #AAPOR @LoveStats 
Optional
Speeding differences not discernable 
Tweet! #AAPOR @LoveStats 
Long 
Short A 
Short B 
Standardized Number of Seconds 
m=0, sd=1, ceiling outliers removed
Drop the worst 5% 
from each dataset 
6.1% of Long 
6.3% of Short A 
5.2% of Short B 
Tweet! #AAPOR @LoveStats
Data 
Equivalence 
Tweet! #AAPOR @LoveStats
Survey Middle: Trips Taken 
  
Tweet! #AAPOR @LoveStats 
80% 
60% 
40% 
20% 
0% 
1 to 3 trips 4 or more trips 
Long Short A Short B
Survey Middle: Airline Aware 
P<.05 P<.05 
Tweet! #AAPOR @LoveStats 
P<.05 
P<.05 
100% 
75% 
50% 
25% 
0% 
Long Short A Short B
Survey End: Like Advertising 
Tweet! #AAPOR @LoveStats 
60% 
50% 
40% 
30% 
20% 
10% 
0% 
Youtube Facebook Flyers Billboards 
Long Short A Short B 
P<.05 
  
Survey End: Privacy Concerns 
     
Tweet! #AAPOR @LoveStats 
60% 
40% 
20% 
0% 
Top2Q91privacy Top2Q91security Top2Q91family Top2Q91home Top2Q91online 
Long ShortB
Survey End: Personality 
      
Tweet! #AAPOR @LoveStats
Respondent 
Engagement 
Tweet! #AAPOR @LoveStats
Same Completion Trend 
Optional 
watch a 
commercial 
Optional 
watch a 
commercial 
Tweet! #AAPOR @LoveStats 
100% 
75% 
50% 
25% 
Long Short A Short B 
Grid
More Words in Verbatim 
    
Tweet! #AAPOR @LoveStats 
60 
40 
20 
0 
Verbatim 1 Verbatim 2 Verbatim 3 Verbatim 4 Verbatim 5 Verbatim 6 
Long ShortA ShortB 
Optional 
P<.05
Possibly Higher Satisfaction 
P<.05   
P<.05 
Tweet! #AAPOR @LoveStats 
100% 
75% 
50% 
25% 
0% 
Easy Boring Long Reco 
Long Short A Short B 
P<.05
More Positive Survey Comments 
Tweet! #AAPOR @LoveStats 
75% 
86% 
93% 
Long Short A Short B 
Percentage of optional survey comments that were positive
Conclusions 
• Not a slam-dunk win 
• Data quality is as good or better 
• Results are equivalent 
• Survey satisfaction is as good or 
better 
Tweet! #AAPOR @LoveStats
Final Thoughts 
• Responders self-select for a 10 or 20 minute 
Tweet! #AAPOR @LoveStats 
experience 
• Differences are in line with sampling error 
• Large portions of most surveys do 
not need to be answered by everyone. 
Plan ahead.
Thank you! 
Annie Pettit 
Chief Research Officer 
Peanut Labs 
annie@peanutlabs.com 
@LoveStats on Twitter 
www.peanutlabs.com 
Tweet! #AAPOR @LoveStats 
: )

Weitere ähnliche Inhalte

Andere mochten auch

Atlanta Copywriter Portfolio/Georgia Dzurica
Atlanta Copywriter Portfolio/Georgia DzuricaAtlanta Copywriter Portfolio/Georgia Dzurica
Atlanta Copywriter Portfolio/Georgia DzuricaGDzurica
 
How to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaireHow to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaireAnnie Pettit, Research Methodologist
 
Wor dzportfolio rev13 2
Wor dzportfolio rev13 2Wor dzportfolio rev13 2
Wor dzportfolio rev13 2GDzurica
 

Andere mochten auch (15)

The Rise and Fall and Rise of Social Media Research #IIEXap14
The Rise and Fall and Rise of Social Media Research #IIEXap14The Rise and Fall and Rise of Social Media Research #IIEXap14
The Rise and Fall and Rise of Social Media Research #IIEXap14
 
VUE-JAN-FEB-2015
VUE-JAN-FEB-2015VUE-JAN-FEB-2015
VUE-JAN-FEB-2015
 
How non-native English speakers respond to English surveys
How non-native English speakers respond to English surveysHow non-native English speakers respond to English surveys
How non-native English speakers respond to English surveys
 
Survey analysis in a nutshell with Jeffrey Henning
Survey analysis in a nutshell with Jeffrey HenningSurvey analysis in a nutshell with Jeffrey Henning
Survey analysis in a nutshell with Jeffrey Henning
 
How to Turn Big Data into Actionable Data IIeX Europe 2016
How to Turn Big Data into Actionable Data IIeX Europe 2016How to Turn Big Data into Actionable Data IIeX Europe 2016
How to Turn Big Data into Actionable Data IIeX Europe 2016
 
Atlanta Copywriter Portfolio/Georgia Dzurica
Atlanta Copywriter Portfolio/Georgia DzuricaAtlanta Copywriter Portfolio/Georgia Dzurica
Atlanta Copywriter Portfolio/Georgia Dzurica
 
Casro 2013: Battle of the Scales
Casro 2013: Battle of the ScalesCasro 2013: Battle of the Scales
Casro 2013: Battle of the Scales
 
Angels and Demons: Is your survey penalizing the right people?
Angels and Demons: Is your survey penalizing the right people?Angels and Demons: Is your survey penalizing the right people?
Angels and Demons: Is your survey penalizing the right people?
 
How to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaireHow to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaire
 
Customer Experience: From Bits and Bytes to Flesh and Blood
Customer Experience: From Bits and Bytes to Flesh and BloodCustomer Experience: From Bits and Bytes to Flesh and Blood
Customer Experience: From Bits and Bytes to Flesh and Blood
 
Multimode Global Scale Usage
Multimode Global Scale UsageMultimode Global Scale Usage
Multimode Global Scale Usage
 
How people talk about us behind our backs
How people talk about us behind our backsHow people talk about us behind our backs
How people talk about us behind our backs
 
Wor dzportfolio rev13 2
Wor dzportfolio rev13 2Wor dzportfolio rev13 2
Wor dzportfolio rev13 2
 
How to Create Census Sampling Targets for Free Using Data Ferret
How to Create Census Sampling Targets for Free Using Data FerretHow to Create Census Sampling Targets for Free Using Data Ferret
How to Create Census Sampling Targets for Free Using Data Ferret
 
VUE-JUNE-2015
VUE-JUNE-2015VUE-JUNE-2015
VUE-JUNE-2015
 

Ähnlich wie Effects of splitting long surveys into two

Peanut Labs Webinar - Survey Analysis In A Nutshell
Peanut Labs Webinar - Survey Analysis In A NutshellPeanut Labs Webinar - Survey Analysis In A Nutshell
Peanut Labs Webinar - Survey Analysis In A NutshellPeanut Labs
 
Session Title A or Session Title B: A/B Testing with Concorde Career Colleges
Session Title A or Session Title B: A/B Testing with Concorde Career CollegesSession Title A or Session Title B: A/B Testing with Concorde Career Colleges
Session Title A or Session Title B: A/B Testing with Concorde Career Collegesemfluence
 
Summit 2013 - Sourcing5: 140 characters or Less - Kasper
Summit 2013 - Sourcing5: 140 characters or Less - KasperSummit 2013 - Sourcing5: 140 characters or Less - Kasper
Summit 2013 - Sourcing5: 140 characters or Less - KasperJobvite
 
Tweets polls surveys
Tweets polls surveysTweets polls surveys
Tweets polls surveysSandra Hines
 
7 Proven Strategies to Maximize Twitter for Your Business
7 Proven Strategies to Maximize Twitter for Your Business7 Proven Strategies to Maximize Twitter for Your Business
7 Proven Strategies to Maximize Twitter for Your BusinessDave Kerpen
 
Effects of Asking for Telephone Numbers on Survey Honesty
Effects of Asking for Telephone Numbers on Survey HonestyEffects of Asking for Telephone Numbers on Survey Honesty
Effects of Asking for Telephone Numbers on Survey HonestyPeanut Labs
 
#Protips for utilizing social media to drive engagement
#Protips for utilizing social media to drive engagement#Protips for utilizing social media to drive engagement
#Protips for utilizing social media to drive engagementWill Patch
 
Vietnamese media consumption differences in south and north
Vietnamese media consumption differences in south and northVietnamese media consumption differences in south and north
Vietnamese media consumption differences in south and northQ&Me Vietnam Market Research
 
The Best Subject Lines of 2015
The Best Subject Lines of 2015The Best Subject Lines of 2015
The Best Subject Lines of 2015Kaitlyn Jones, MBA
 
The Best Subject Lines of 2015
The Best Subject Lines of 2015The Best Subject Lines of 2015
The Best Subject Lines of 2015Informz
 
How to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaireHow to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnairePeanut Labs
 
Turning numbers into data. workshop sample slides
Turning numbers into data. workshop sample slidesTurning numbers into data. workshop sample slides
Turning numbers into data. workshop sample slidesKeith Meadows
 
Assessments 2,3,4,5,6,7,8 marshall sponder
Assessments 2,3,4,5,6,7,8 marshall sponderAssessments 2,3,4,5,6,7,8 marshall sponder
Assessments 2,3,4,5,6,7,8 marshall sponderMarshall Sponder
 
Chat App Usage In Thailand
Chat App Usage In ThailandChat App Usage In Thailand
Chat App Usage In ThailandDI Marketing
 
How Cool Vendors stay cool
How Cool Vendors stay coolHow Cool Vendors stay cool
How Cool Vendors stay coolDuncan Chapple
 
PollEverywhere PPT
PollEverywhere PPTPollEverywhere PPT
PollEverywhere PPTKrista Leh
 

Ähnlich wie Effects of splitting long surveys into two (20)

Peanut Labs Webinar - Survey Analysis In A Nutshell
Peanut Labs Webinar - Survey Analysis In A NutshellPeanut Labs Webinar - Survey Analysis In A Nutshell
Peanut Labs Webinar - Survey Analysis In A Nutshell
 
Session Title A or Session Title B: A/B Testing with Concorde Career Colleges
Session Title A or Session Title B: A/B Testing with Concorde Career CollegesSession Title A or Session Title B: A/B Testing with Concorde Career Colleges
Session Title A or Session Title B: A/B Testing with Concorde Career Colleges
 
Summit 2013 - Sourcing5: 140 characters or Less - Kasper
Summit 2013 - Sourcing5: 140 characters or Less - KasperSummit 2013 - Sourcing5: 140 characters or Less - Kasper
Summit 2013 - Sourcing5: 140 characters or Less - Kasper
 
Tweets polls surveys
Tweets polls surveysTweets polls surveys
Tweets polls surveys
 
7 Proven Strategies to Maximize Twitter for Your Business
7 Proven Strategies to Maximize Twitter for Your Business7 Proven Strategies to Maximize Twitter for Your Business
7 Proven Strategies to Maximize Twitter for Your Business
 
Top 10 Tips for Your Split Test Program by Justin Rondeau
Top 10 Tips for Your Split Test Program by Justin RondeauTop 10 Tips for Your Split Test Program by Justin Rondeau
Top 10 Tips for Your Split Test Program by Justin Rondeau
 
Effects of Asking for Telephone Numbers on Survey Honesty
Effects of Asking for Telephone Numbers on Survey HonestyEffects of Asking for Telephone Numbers on Survey Honesty
Effects of Asking for Telephone Numbers on Survey Honesty
 
#Protips for utilizing social media to drive engagement
#Protips for utilizing social media to drive engagement#Protips for utilizing social media to drive engagement
#Protips for utilizing social media to drive engagement
 
Vietnamese media consumption differences in south and north
Vietnamese media consumption differences in south and northVietnamese media consumption differences in south and north
Vietnamese media consumption differences in south and north
 
Vietnamese media consumption South vs. North
Vietnamese media consumption South vs. North Vietnamese media consumption South vs. North
Vietnamese media consumption South vs. North
 
Social Listening in the Travel Category
Social Listening in the Travel CategorySocial Listening in the Travel Category
Social Listening in the Travel Category
 
The Best Subject Lines of 2015
The Best Subject Lines of 2015The Best Subject Lines of 2015
The Best Subject Lines of 2015
 
The Best Subject Lines of 2015
The Best Subject Lines of 2015The Best Subject Lines of 2015
The Best Subject Lines of 2015
 
How to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaireHow to seamlessly integrate data quality measures into your questionnaire
How to seamlessly integrate data quality measures into your questionnaire
 
Turning numbers into data. workshop sample slides
Turning numbers into data. workshop sample slidesTurning numbers into data. workshop sample slides
Turning numbers into data. workshop sample slides
 
Assessments 2,3,4,5,6,7,8 marshall sponder
Assessments 2,3,4,5,6,7,8 marshall sponderAssessments 2,3,4,5,6,7,8 marshall sponder
Assessments 2,3,4,5,6,7,8 marshall sponder
 
Chat App Usage In Thailand
Chat App Usage In ThailandChat App Usage In Thailand
Chat App Usage In Thailand
 
Maeghan Smulders - Customer Interviews
Maeghan Smulders - Customer InterviewsMaeghan Smulders - Customer Interviews
Maeghan Smulders - Customer Interviews
 
How Cool Vendors stay cool
How Cool Vendors stay coolHow Cool Vendors stay cool
How Cool Vendors stay cool
 
PollEverywhere PPT
PollEverywhere PPTPollEverywhere PPT
PollEverywhere PPT
 

Mehr von Annie Pettit, Research Methodologist

A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...
A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...
A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...Annie Pettit, Research Methodologist
 
Social Media Data from Research to Customers - Casro Digital in San Antonio, ...
Social Media Data from Research to Customers - Casro Digital in San Antonio, ...Social Media Data from Research to Customers - Casro Digital in San Antonio, ...
Social Media Data from Research to Customers - Casro Digital in San Antonio, ...Annie Pettit, Research Methodologist
 

Mehr von Annie Pettit, Research Methodologist (13)

AI and Voice Search and Chatbots, oh my!
AI and Voice Search and Chatbots, oh my!AI and Voice Search and Chatbots, oh my!
AI and Voice Search and Chatbots, oh my!
 
Cognitive Biases for Marketers
Cognitive Biases for MarketersCognitive Biases for Marketers
Cognitive Biases for Marketers
 
Scientific rigour is worth melting for MRS Annual Conference 2016
Scientific rigour is worth melting for MRS Annual Conference 2016Scientific rigour is worth melting for MRS Annual Conference 2016
Scientific rigour is worth melting for MRS Annual Conference 2016
 
NewMR 2016 presents: 9 Big Applications of Big Data
NewMR 2016 presents: 9 Big Applications of Big DataNewMR 2016 presents: 9 Big Applications of Big Data
NewMR 2016 presents: 9 Big Applications of Big Data
 
Blasting 10 Big Data Myths with 10 Panel Data Examples
Blasting 10 Big Data Myths with 10 Panel Data ExamplesBlasting 10 Big Data Myths with 10 Panel Data Examples
Blasting 10 Big Data Myths with 10 Panel Data Examples
 
9 Tips For Creating a Great Numeric Question
9 Tips For Creating a Great Numeric Question9 Tips For Creating a Great Numeric Question
9 Tips For Creating a Great Numeric Question
 
A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...
A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...
A Tale of Two Surveys: How using real words instead of mumbo jumbo affects su...
 
Using social media data for new product development
Using social media data for new product developmentUsing social media data for new product development
Using social media data for new product development
 
Social Media Data from Research to Customers - Casro Digital in San Antonio, ...
Social Media Data from Research to Customers - Casro Digital in San Antonio, ...Social Media Data from Research to Customers - Casro Digital in San Antonio, ...
Social Media Data from Research to Customers - Casro Digital in San Antonio, ...
 
Behavioural Economics: New or Not?
Behavioural Economics: New or Not?Behavioural Economics: New or Not?
Behavioural Economics: New or Not?
 
20 ways to use social media research
20 ways to use social media research20 ways to use social media research
20 ways to use social media research
 
Advanced Analytics with Social Media Data
Advanced Analytics with Social Media DataAdvanced Analytics with Social Media Data
Advanced Analytics with Social Media Data
 
AMA MRA Social Media Research Myths
AMA MRA Social Media Research MythsAMA MRA Social Media Research Myths
AMA MRA Social Media Research Myths
 

Kürzlich hochgeladen

YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 

Kürzlich hochgeladen (17)

YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 

Effects of splitting long surveys into two

  • 1. Effects of Splitting Long Surveys Into Two Annie Pettit Chief Research Officer Tweet! #AAPOR @LoveStats
  • 2. 90 77 79 73 72 62 43 40 34 31 21 14 36 28 25 21 15 9 http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/ Tweet! #AAPOR @LoveStats
  • 3. 50% of surveys are >=15 minutes 5 10 15 20 25 30 35 40 45 50 55 60+ Tweet! #AAPOR @LoveStats Source: DIY Sampling Panel
  • 4. Tweet! #AAPOR @LoveStats 12 Minute Survey N=300 Split incentive Method 12 Minute Survey N=300 Split incentive 20 Minute Survey N=600 Full incentive
  • 5. Problem: Nothing is directly Comparable • People choose whether to participate • Each version is different – Different length – Different topics – Different leading/priming questions Tweet! #AAPOR @LoveStats
  • 6. Tweet! #AAPOR @LoveStats Agenda 1. Data quality 2. Data equivalence 3. Responder satisfaction 11/6/2014 6
  • 7. Data Quality 11/6/2014 7 Tweet! #AAPOR @LoveStats
  • 8. Fewer Red Herrings     Tweet! #AAPOR @LoveStats 2 fake hotels 2 fake airlines 2 fake loyalty programs
  • 9. Straightlining   P<.05 Tweet! #AAPOR @LoveStats 50% 40% 30% 20% 10% 0% Fail Straight 1 Fail Straight 6 Fail Straight 10 Fail Straight 64 Fail Straight 66 Fail Straight 85 Fail Straight 91 Fail Straight 93 Fail Straight 104 Long Short A Short B
  • 10. Follow Instructions Better   Did not follow instruction to “choose only one” Tweet! #AAPOR @LoveStats
  • 11. Better Open Ends      Tweet! #AAPOR @LoveStats Optional
  • 12. Speeding differences not discernable Tweet! #AAPOR @LoveStats Long Short A Short B Standardized Number of Seconds m=0, sd=1, ceiling outliers removed
  • 13. Drop the worst 5% from each dataset 6.1% of Long 6.3% of Short A 5.2% of Short B Tweet! #AAPOR @LoveStats
  • 14. Data Equivalence Tweet! #AAPOR @LoveStats
  • 15. Survey Middle: Trips Taken   Tweet! #AAPOR @LoveStats 80% 60% 40% 20% 0% 1 to 3 trips 4 or more trips Long Short A Short B
  • 16. Survey Middle: Airline Aware P<.05 P<.05 Tweet! #AAPOR @LoveStats P<.05 P<.05 100% 75% 50% 25% 0% Long Short A Short B
  • 17. Survey End: Like Advertising Tweet! #AAPOR @LoveStats 60% 50% 40% 30% 20% 10% 0% Youtube Facebook Flyers Billboards Long Short A Short B P<.05   
  • 18. Survey End: Privacy Concerns      Tweet! #AAPOR @LoveStats 60% 40% 20% 0% Top2Q91privacy Top2Q91security Top2Q91family Top2Q91home Top2Q91online Long ShortB
  • 19. Survey End: Personality       Tweet! #AAPOR @LoveStats
  • 20. Respondent Engagement Tweet! #AAPOR @LoveStats
  • 21. Same Completion Trend Optional watch a commercial Optional watch a commercial Tweet! #AAPOR @LoveStats 100% 75% 50% 25% Long Short A Short B Grid
  • 22. More Words in Verbatim     Tweet! #AAPOR @LoveStats 60 40 20 0 Verbatim 1 Verbatim 2 Verbatim 3 Verbatim 4 Verbatim 5 Verbatim 6 Long ShortA ShortB Optional P<.05
  • 23. Possibly Higher Satisfaction P<.05   P<.05 Tweet! #AAPOR @LoveStats 100% 75% 50% 25% 0% Easy Boring Long Reco Long Short A Short B P<.05
  • 24. More Positive Survey Comments Tweet! #AAPOR @LoveStats 75% 86% 93% Long Short A Short B Percentage of optional survey comments that were positive
  • 25. Conclusions • Not a slam-dunk win • Data quality is as good or better • Results are equivalent • Survey satisfaction is as good or better Tweet! #AAPOR @LoveStats
  • 26. Final Thoughts • Responders self-select for a 10 or 20 minute Tweet! #AAPOR @LoveStats experience • Differences are in line with sampling error • Large portions of most surveys do not need to be answered by everyone. Plan ahead.
  • 27. Thank you! Annie Pettit Chief Research Officer Peanut Labs annie@peanutlabs.com @LoveStats on Twitter www.peanutlabs.com Tweet! #AAPOR @LoveStats : )