Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
principle of accounts sba
principle of accounts sba
Loading in …3
×
1 of 12

Does Sample Source Matter? [Part 2]

1

Share

Download to read offline

A comparison of some leading sample sources including river sample and router sample.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Does Sample Source Matter? [Part 2]

  1. 1. Does the Sample Source Matter? Wave 2 of a comparison of leading sample sources Sept 07, 2012
  2. 2. Sample is the soil and water of research. The quality of it determines the value of what you produce.
  3. 3. We compared other sample sources that VC regularly uses We conducted a five minute survey April 12, 2012 with a sample of about 500 people each, using sample from Vision Critical’s Springboard America, and four of the leading sample providers that we have anonymised as “Panel B”, “River C”, “Panel D” and “River E”. The questionnaire included items from the US 2010 General Social Survey, as well as a questions about snack consumption and a concept test for a snack food. We counted the number of statistically significant differences between the panels (95% CI)
  4. 4. Methodology cont. The questionnaire included items from the US 2010 General Social Survey, as well as a questions about snack consumption and a concept test for a snack food. We counted the number of statistically significant differences between the panels (95% CI), and looked at drop-out rates, straight lining, and the use of “don’t know”.
  5. 5. Sample Quality: Dropout rate during the survey Drop-outs as a percent of all who started the survey Nov-11 Apr-12 20% 19% 18% 16% 16% 14% 12% 10% 10% 8% 9% 8% 7% 6% 5% 5% 5% 4% 4% 4% 2% 2% 0 0 0% SBA - SBA - Panel B Lightspeed Panel C SSI Research Panel D River United Panel E Targeted Routed Now Sample
  6. 6. Data quality: straight-lining 10% 9% 7.9% 7.8% 8% 7% 6.5% 6% 5.5% 5.6% 5% 4.8% 3.9% 4% 3.1% 3% 2.7% 2.1% 2% NA 1% 0.7% 0.5% 0% SBA Panel B SBA Routed Lightspeed Panel C SSI Panel D Research SB River Panel E uSamp Now
  7. 7. Data quality: use of “don’t know” Nov-11 Apr-12 10% 9% 8% 7% 6% 6% 6% 6% 5% 5% 4% 4% 4% 4% 4% 4% 4% 4% 3% 3% 2% 1% 0 0 0% SBA - SBA - Panel B Lightspeed Panel C SSI Research SB River Panel E Panel D uSamp Targeted Routed Now Average percent who choose the option "don’t know", "can't choose", "not sure", "prefer not to say".
  8. 8. Differences between SBA and other panels in how people answered all questions 10% 9% 8% 7% 7% 6% Expected 5% 4% by chance 4% 4% 3% 3% 2% 2% 2% 1% 0% Panel B SBA Routed Lightspeed Panel C SSI Panel D Research Panel E uSamp River Now Count of the number of statistically significant differences (95% CI) as a percent of the total number of questions
  9. 9. Differences between SBA and other panels in how people answered all questions While, for the most part, these differences appear within the realm of random, there were some differences that seem to be very real. For example, in both April and November Panel B respondents were less likely to purchase snack products and had lower purchase intent scores for a snack concept.
  10. 10. Differences between panels in how people answered the purchase intent question Nov-11 Apr-12 50% 46% 44% 44% 45% 41% 40% 39% 40% 34% 33% 35% 31% 30% 30% 29% 25% 20% 20% 15% 10% 5% 0 0 0% SBA - SBA - Lightspeed Panel C Research Panel B SSI Panel D River Panel E United Targeted Routed Now Sample Which statement below best describes how likely you would be purchase this snack if it were available today?—top 2 box: definitely and probably
  11. 11. Conclusions Sample matters. A lot. Springboard America has fewer drop-outs, notably better data quality, and is “right down the middle” when it comes to comparison to the other panels on the findings, unlike some panels which show real variation—particularly on the concept test. SBA also shows consistency in how people answered, as illustrated by reaction to the concept.
  12. 12. Conclusions The same cannot be said for the other panels, however. They showed considerable variation between time 1 and time 2 in how they reacted to the snack concept, which was a non-trendy concept. This may be a concern for tracking. I will dig deeper into the comparison between waves in another report.

×