SlideShare ist ein Scribd-Unternehmen logo
1 von 143
Downloaden Sie, um offline zu lesen
Practical 
Lean Analytics 
Growthcon 
October 24, 2014 
@acroll
Kevin Costner is a lousy entrepreneur. 
Don’t sell what you can make. Make what you can sell.
The core of Lean 
is iteration.
Most startups don’t know what they’ll 
be when they grow up. 
Hotmail 
was a 
database 
company 
Flickr 
was going to 
be an MMO 
Twitter 
was a 
podcasting 
company 
Autodesk 
made 
desktop 
automation 
Paypal 
first built for 
Palmpilots 
Freshbooks 
was invoicing 
for a web 
design firm 
Wikipedia 
was to be 
written by 
experts only 
Mitel 
was a 
lawnmower 
company
Waterfall, agile, and lean 
(Why the old ways don’t work.)
Waterfall approach 
You know the problem and the solution.
Known set of 
requirements 
Known ways to satisfy 
them 
Spec Build Test Launch
Agile methodologies 
Know the problem, find the solution
Known set of 
requirements 
Unclear how to satisfy 
them 
Problem Build Test Viable? Launch 
statement 
Sprints 
Adjust 
Unknown set of
Lean approach 
First, know that you don’t know.
Product/market 
hypothesis Trial startup 
Possible problem 
space 
Product/ 
market 
hypothesis 
Trial startup 
Product/ 
market 
hypothesis 
Trial startup 
Trial startup 
Product/market 
hypothesis 
You are 
here 
PIVOT
Why now? 
First: High rate of change of digital 
technologies & channels.
Arbitron and radio data
Times a song in “heavy 
rotation” is played daily 
30 
15 
0 
6 26 
2007 2012
Why now? 
Second: It’s no longer about whether 
you can build it—it’s about whether 
anyone will care.
The Attention Economy 
“What information consumes 
is rather obvious: it consumes 
the attention of its recipients. 
Hence a wealth of information 
creates a poverty of attention, and a 
need to allocate that attention efficiently 
among the overabundance of 
information sources that might 
consume it.” 
(Computers, Communications and the Public Interest, pages 40-41, 
Herbert Simon Martin Greenberger, ed., The Johns Hopkins Press, 1971.)
Lit motors tests the risky part
Unfortunately, it is hard to be honest 
with ourselves.
Everyone’s idea is 
the best right? 
People love 
this part! 
(but that’s not always 
a good thing) 
No data, no 
learning. 
This is where 
things fall apart.
Analytics can help.
Analytics is the measurement of 
movement towards your business 
goals.
In a startup, the purpose of analytics is 
to iterate to product/market fit 
before the money runs out.
Some fundamentals.
A good metric is: 
Understandable 
If you’re busy 
explaining the 
data, you won’t 
be busy acting 
on it. 
Comparative 
Comparison is 
context. 
A ratio or rate 
The only way to 
measure 
change and roll 
up the tension 
between two 
metrics (MPH) 
Behavior 
changing 
What will you 
do differently 
based on the 
results you 
collect?
The 
simplest 
rule 
If a metric won’t change how 
you behave, it’s a 
bad 
metric. 
h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself. 
Acquisition 
Hybrid 
Loyalty 
You are 
just like 
70% 
of retailers 
20% 
of retailers 
10% 
of retailers 
Customers that 
buy >1x in 90d 
Your customers 
will buy from you 
Once 
2-2.5 
per year 
>2.5 
per year 
Then you are 
in this mode 
1-15% 
15-30% 
>30% 
Focus on 
Low acquisition 
cost, high checkout 
Increasing return 
rates, market share 
Loyalty, selection, 
inventory size 
(Thanks to Kevin Hillstrom for this.)
Qualitative 
Unstructured, anecdotal, 
revealing, hard to 
aggregate, often too 
positive & reassuring. 
Warm and fuzzy. 
Quantitative 
Numbers and stats. 
Hard facts, less insight, 
easier to analyze; often 
sour and disappointing. 
Cold and hard.
Exploratory 
Speculative. Tries to find 
unexpected or 
interesting insights. 
Source of unfair 
advantages. 
Cool. 
Reporting 
Predictable. Keeps you 
abreast of the normal, 
day-to-day operations. 
Can be managed by 
exception. 
Necessary.
Rumsfeld on Analytics 
Things we 
know 
don’t 
know 
(Or rather, Avinash Kaushik channeling Rumsfeld) 
we know Are facts which may be wrong and 
should be checked against data. 
we don’t 
know 
Are questions we can answer by 
reporting, which we should baseline 
& automate. 
we know 
Are intuition which we should 
quantify and teach to improve 
effectiveness, efficiency. 
we don’t 
know 
Are exploration which is where 
unfair advantage and interesting 
epiphanies live.
Slicing and dicing data 
Feb Mar Apr May 
5,000 
Active users 
0 
Jan 
Cohort: 
Comparison of 
similar groups 
along a timeline. 
(this is the April cohort) 
A/B test: 
Changing one thing 
(i.e. color) and 
measuring the 
result (i.e. revenue.) 
Multivariate 
analysis 
Changing several 
things at once to 
see which correlates 
with a result. 
☀☁☀☁ 
Segment: 
Cross-sectional 
comparison of all 
people divided by 
some attribute (age, 
gender, etc.) 
☀ 
☁
Which of these two companies 
is doing better?
January February March April May 
Is this company Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 
growing or stagnating? 
Cohort 1 2 3 4 5 
January $5 $3 $2 $1 $0.5 
February $6 $4 $2 $1 
March $7 $6 $5 
April $8 $7 
May $9 
How about 
this one?
Cohort 1 2 3 4 5 
January $5 $3 $2 $1 $0.5 
February $6 $4 $2 $1 
March $7 $6 $5 
April $8 $7 
May $9 
Averages $7 $5 $3 $1 $0.5 
Look at the 
same data 
in cohorts
Lagging 
Historical. Shows you 
how you’re doing; 
reports the news. 
Example: sales. 
Explaining the 
past. 
Leading 
Forward-looking. 
Number today that 
predicts tomorrow; 
reports the news. 
Example: pipeline. 
Predicting the 
future.
Some examples 
A Facebook user reaching 7 friends within 10 days of signing up 
(Chamath Palihapitiya) 
If someone comes back to Zynga a day after signing up for a game, 
they’ll probably become an engaged, paying user (Nabeel Hyatt) 
A Dropbox user who puts at least one file in one folder on one device 
(ChenLi Wang) 
Twitter user following a certain number of people, and a certain 
percentage of those people following the user back (Josh Elman) 
A LinkedIn user getting to X connections in Y days (Elliot Schmukler) 
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Which means it’s time to talk 
about correlation.
10000 
1000 
100 
10 
1 
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec 
Ice cream consumption Drownings
Correlated 
Two variables that are 
related (but may be 
dependent on 
something else.) 
Ice cream & 
drowning. 
Causal 
An independent variable 
that directly impacts a 
dependent one. 
Summertime & 
drowning.
A leading, causal metric 
is a superpower. 
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Why is Nigerian spam so badly 
written?
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages 
emailed; they expect to land 2 or 3 “Mugu” (fools) each week. 
One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” 
“By sending an email that repels all but the most gullible,” says [Microsoft Researcher 
Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts 
the true to false positive ratio in his favor.” 
This would be horribly 
inefficient since 
humans are involved. 
Good language (10% conversion) 
Not-gullible (.07% conversion) 
Aunshul Rege of Rutgers University, USA in 2009 
1000 emails 
Bad language (0.1% conversion) 
1-2 responses 
Gullible (70% conversion) 
1 fool and their money, parted. 
1000 emails 
100 responses 
1 fool and their money, parted.
Turns out the word “Nigeria” is the best 
way to identify promising prospects.
Nigerian spammers 
really understand their target market. 
They see past vanity metrics.
The Lean Analytics framework.
Sustainable growth comes through the 
actions of your customers. 
- Eric Ries
Eric’s three engines of growth 
Virality 
Make people 
invite friends. 
How many they 
tell, how fast they 
tell them. 
Price 
Spend money to 
get customers. 
Customers are 
worth more than 
they cost. 
Stickiness 
Keep people 
coming back. 
Approach 
Get customers 
faster than you 
lose them. 
Math that 
matters
Dave’s Pirate Metrics 
AARRR Acquisition 
How do your users become aware of you? 
SEO, SEM, widgets, email, PR, campaigns, blogs ... 
Activation 
Do drive-by visitors subscribe, use, etc? 
Features, design, tone, compensation, affirmation ... 
Retention 
Does a one-time user become engaged? 
Notifications, alerts, reminders, emails, updates... 
Revenue 
Do you make money from user activity? 
Transactions, clicks, subscriptions, DLC, analytics... 
Referral 
Do users promote your product? 
Email, widgets, campaigns, likes, RTs, affiliates...
Gate 
Stage 
EMPATHY I’ve found a real, poorly-met need that a 
reachable market faces. 
STICKINESS I’ve figured out how to solve the problem in a 
way they will keep using and pay for. 
VIRALITY I’ve found ways to get them to tell their friends, 
either intrinsically or through incentives. 
REVENUE The users and features fuel growth organically 
and artificially. 
SCALE I’ve found a sustainable, scalable business with 
the right margins in a healthy ecosystem. 
The five stages
Empathy stage: 
Localmind hacks Twitter 
Needed to find out if a core assumption—strangers answering 
questions—was valid. 
Ran Twitter experiment instead of writing code 
Asked senders of geolocated Tweets from Times Square random 
questions; counted response rate 
Conclusion: high enough to proceed
LikeBright’s mechanical turk 
Used Mechanical Turk, Google Voice to speak w/ 
100 single women; paid $2. The interviews 
lasted typically around 10-15 minutes. 
Simple interview script with open-ended 
questions, since he was digging into the problem 
validation stage of his startup. 
Founder Nick Soman: “I was amazed at the 
feedback I got. We were able to speak with one 
hundred single women that met our criteria in 
four hours on one evening.” 
Went back to TechStars and got accepted. 
LikeBright’s website is now live with a 50% 
female user base, and recently raised a round of 
funding. 
“Since that first foray into interviewing customers, 
I’ve probably spoken with over a thousand 
people through Mechanical Turk,”
How to avoid leading the witness 
Avoid biased wording, preconceptions, or 
a giveaway appearance. Word your 
surveys carefully to be neutral. 
Get them to purchase. Ask them to pay. Demand real 
Ask “why” several times. Leave lingering, uncomfortable pauses 
Don’t tip your hand 
Make the question real 
Keep digging 
Look for other clues Have a colleague make notes of when they react, or of their body
Stickiness stage: 
qidiq streamlines invites 
Survey owner adds recipient to group 
Survey owner asks question 
Recipient reads survey question 
Recipient responds to question 
Recipient sees survey results 
(Later, if needed…) 
Recipient visits site; no password! 
Recipient does password recovery 
One-time link sent to email 
Recipient creates password 
Recipient can edit profile, etc. 
Survey owner adds recipient to group 
Survey owner asks question 
Recipient gets invite 
Recipient installs mobile app 
Recipient creates account, profile 
Recipient can edit profile, etc. 
Recipient reads survey question 
Recipient responds to question 
Recipient sees survey results 
10-25% RESPONSE RATE 
70-90% RESPONSE RATE
1200 
1000 
800 
600 
400 
200 
0 
January February 
1 2 3 4 5 6 7 8 9 
Days since last engagement 
25000 
20000 
15000 
10000 
5000 
0 
Disengaged 
(>10 days) 
Number of users 
A better approach to engagement 
This is a 
good thing.
Who is worth more? 
A Lifetime: 
B Lifetime: 
Today 
$200 
$200 
Roberto Medri, Etsy 
Visits
Virality stage: Timehop focuses 
on content sharing 
Focused on percent of daily active users that share their content 
Aiming for 20-30% of DAU sharing 
“All that matters now is virality. Everything else—be 
it press, publicity stunts or something else—is like 
pushing a rock up a mountain: it will never scale. 
But being viral will.” 
- Jonathan Wegener, co-founder
------------------------------------------------------ 
Get your free private email at http://www.hotmail.com 
------------------------------------------------------
v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0 
http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ 
Viral coefficient
Or simpler 
x - > 1 
Users Viral 
coefficient 
Churn & 
abandonment
Revenue stage: Backupify’s 
Customer Acquisition Payback 
Initially focused on site visitors 
Then focused on trials 
Then switched to signups 
Today, MRR 
In early 2010, CAC was $243 and ARPU was only $39 
Pivoted to target business users 
CLV-to-CAC today is 5-6x 
Now they track Customer Acquisition Payback 
Target is less than 12 months
Scale stage: 
Incremental order cost 
Marginal costs 
Fixed costs
Six business model archetypes. 
E-commerce SaaS Mobile Media 
app 
User-gen 
content 
2-sided 
market 
The business you’re in
(Which means eye 
charts like these.) 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Acquisition 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Activation 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Retention 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Revenue 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Revenue 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Model + Stage = One Metric That Matters. 
The business you’re in 
E-Com SaaS Mobile 2-Sided Media UCG 
One Metric 
That Matters. 
Empathy 
Stickiness 
Virality 
Revenue 
Scale 
The stage you’re at
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric 
addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics 
SaaS-based SEO toolkit in the scale stage. Focused on net adds. 
Was a marketing campaign successful? 
Were customer complaints lowered? 
Was a product upgrade valuable? 
Net adds up: 
Can we acquire more valuable customers? 
What product features can increase engagement? 
Can we improve customer support? 
Net adds flat: 
Are the new customers not the right segment? 
Did a marketing campaign fail? 
Did a product upgrade fail somehow? 
Is customer support falling apart? 
Net adds down:
Metrics are like squeeze toys. 
http://www.flickr.com/photos/connortarter/4791605202/
Empathy 
Stickiness 
Virality 
Revenue 
Scale 
E-commerce 
Mobile 
app 
User-gen 
content 
SaaS Media 
2-sided 
market 
Interviews; qualitative results; quantitative scoring; surveys 
Loyalty, 
conversion 
CAC, shares, 
reactivation 
(Money from transactions) 
Transaction, 
CLV 
Affiliates, 
white-label 
Engagement, 
churn 
Inherent 
virality, CAC 
(Money from active users) 
Upselling, 
CAC, CLV 
API, magic #, 
mktplace 
Content, 
spam 
Invites, 
sharing 
(Money from ad clicks) 
Ads, 
donations 
Analytics, 
user data 
Inventory, 
listings 
SEM, sharing 
Transactions, 
commission 
Other 
verticals 
Downloads, 
churn, virality 
WoM, app 
ratings, CAC 
CLV, 
ARPDAU 
Spinoffs, 
publishers 
Traffic, visits, 
returns 
Content 
virality, SEM 
CPE, affiliate 
%, eyeballs 
Syndication, 
licenses
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its 
customers every year. 
Is this good or bad?
Baseline: 
2-5% monthly churn 
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s 
on the job. 
• Get below a 5% monthly churn rate before you know you’ve got a 
business that’s ready to grow (Mark MacLeod) and around 2% 
before you really step on the gas (David Skok) 
• Last-ditch appeals and reactivation can have a big impact. 
Facebook’s “don’t leave” reduces attrition by 7%.
Not knowing what normal is 
makes you do unwise things.
Baseline: 
5-7% growth a week 
“A good growth rate during YC 
is 5-7% a week,” he says. “If 
you can hit 10% a week you're 
doing exceptionally well. If you 
can only manage 1%, it's a sign 
you haven't yet figured out 
what you're doing.” At revenue 
stage, measure growth in 
revenue. Before that, measure 
growth in active users. 
Paul Graham, Y Combinator 
• Are there enough people who really care 
enough to sustain a 5% growth rate? 
• Don’t strive for a 5% growth at the expense 
of really understanding your customers 
and building a meaningful solution 
• Once you’re a pre-revenue startup at or 
near product/market fit, you should have 
5% growth of active users each week 
• Once you’re generating revenues, they 
should grow at 5% a week
Baseline: 
10% visitor engagement/day 
30% of users/month use web or mobile app 
10% of users/day use web or mobile app 
1%of users/day use it concurrently 
Fred Wilson’s social ratios
Baseline: 
Calculating customer lifetime 
25% 
5% 
monthly churn 
monthly churn 
100/25=4 
100/5=20 
The average 
The average 
customer lasts 
customer lasts 
4 months 
20 months 
2% 
monthly churn 
100/2=50 
The average 
customer lasts 
50 months
Baseline: 
CAC under 1/3 of CLV 
• CLV is wrong. CAC Is probably wrong, too. 
• Time kills all plans: It’ll take a long time to find 
out whether your churn and revenue projections 
are right 
• Cashflow: You’re basically “loaning” the 
customer money between acquisition and CLV. 
• It keeps you honest: Limiting yourself to a 
CAC of only a third of your CLV will forces you 
to verify costs sooner. 
Lifetime of 20 mo. 
$30/mo. per 
customer 
$600 CLV 
1/3 spend 
$200 CAC 
Now segment 
those users!
Etsy 
• Online store for creative types, founded 2005 
• $525M Gross Merchandise Sales in 2011, with 
19,000,000 members and 800,000 active 
shops offering 15,000,000 items for sale 
• 1.4B pageviews per month ~2M iPhone app 
downloads 
• Thin revenues: Etsy makes only $0.20 or 3.5% 
margin 
• Heavy focus on Customer Lifetime Value (buyer 
and seller) 
• Actually residual lifetime value; they take this 
pretty seriously.
Etsy 
• The best customers to target are 
• Recent high-profile customers 
• Old-time best customers about to 
churn or just churned 
• Tiered campaigns 
• Bronze/silver customers: reinforcement, 
nudges 
• Gold customers: premium services 
• Platinum customers: recognition 
• What they watch: 
• Growth of individual product categories 
• Time to first sale by a user 
• Average order value 
• Percentage of visits that convert to a 
sale 
• Percentage of return buyers 
• Distinct sellers within a product 
category 
• Time-to-first-sale and average order 
value by product category 
Roberto Medri, Etsy
The Lean Analytics cycle
Pick a KPI Draw a line 
Draw a new line 
Pivot or 
give up 
Try again 
Success! 
Did we move the 
needle? 
Measure 
the results 
Design a test 
Make changes 
in production 
Find a potential 
improvement 
With data: 
find a 
commonality 
Without data: 
make a good 
guess 
Hypothesis
Do AirBnB hosts 
get more business 
if their property is 
professionally 
photographed?
Gut instinct (hypothesis) 
Professional photography helps AirBnB’s business 
Candidate solution (MVP) 
20 field photographers posing as employees 
Measure the results 
Compare photographed listings to a control group 
Make a decision 
Launch photography as a new feature for all hosts
5,000 shoots per month 
by February 2012
Hang on a second.
REALLY? 
Gut instinct (hypothesis) 
Professional photography helps AirBnB’s business
Pick a KPI Draw a line 
Draw a new line 
Pivot or 
give up 
Try again 
Success! 
Did we move the 
needle? 
Measure 
the results 
Design a test 
Make changes 
in production 
Find a potential 
improvement 
With data: 
find a 
commonality 
Without data: 
make a good 
guess 
Hypothesis
“Gee, those 
houses that do 
well look really 
nice.” 
Maybe it’s the 
camera. 
With data: 
find a commonality 
“Computer: What 
do all the 
highly rented 
houses have in 
common?” 
Camera model. 
Without data: make a 
good guess
Circle of Moms: Not enough engagement 
• Too few people were 
actually using the 
product 
• Less than 20% of any 
circles had any activity 
after their initial creation 
• A few million monthly 
uniques from 10M 
registered users, but no 
sustained traction 
• They found moms were far more engaged 
• Their messages to one another were on average 50% longer 
• They were 115% more likely to attach a picture to a post they wrote 
• They were 110% more likely to engage in a threaded (i.e. deep) 
conversation 
• Circle owners’ friends were 50% more likely to engage with the circle 
• They were 75% more likely to click on Facebook notifications 
• They were 180% more likely to click on Facebook news feed items 
• They were 60% more likely to accept invitations to the app 
• Pivoted to the new market, including a name change 
• By late 2009, 4.5M users and strong engagement 
• Sold to Sugar, inc. in early 2012
Landing page design A/B testing 
Cohort analysis General analytics 
URL shortening 
Funnel analytics 
Influencer Marketing 
Publisher analytics 
SaaS analytics 
Gaming analytics 
User analytics Spying on users 
User interaction Customer User segmentation satisfaction KPI dashboards
Growth hacking 
(is a word you should hate but will hear a lot about.)
Growth hacking, demystified. 
Find 
correlation 
Test 
causality 
Optimize the 
causal factor 
Pick a metric 
to change
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ 
Is social action a leading 
indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ 
Is mobile use?
Guerrilla 
marketing 
Data-driven 
learning 
GROWTH 
HACKING 
Subversiveness
AirBnB and Craigslist
Take baby steps.
Netflix
Tesla 
http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character 
limit isn’t arbitrary. It’s 
constrained by the size 
http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ 
sms_screen_twitter_activity_stream_270x405.png
http://www.flickr.com/photos/bootbearwdc/1243690099/ 
Think subversively.
To summarize:
1. Define your business model
2. Draw a 
system 
diagram
3. Decide what 
stage you’re at
4. Identify the One Metric That Matters 
(usually the one that is most broken)
5. Use the cycle to 
experiment until 
you’ve achieved 
the desired result.
6. Set up monitoring for this metric in 
case it breaks, and choose a new 
OMTM
Conclusions
“The most important figures that one 
needs for management are unknown 
or unknowable, but successful 
management must nevertheless take 
account of them.” 
Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES 
HAD TAKEN 
BATHS BEFORE.
Once, a leader convinced others 
in the absence of data.
Now, a leader knows 
what questions to ask.
Ben Yoskovitz 
byosko@gmail.com 
@byosko 
Alistair Croll 
acroll@gmail.com 
@acroll
The mobile app! 
customer lifecycle! 
Ratings 
Reviews 
Search 
Leaderboards 
Purchases 
App store! 
App sales 
Downloads 
Installs 
Play 
Disengagement 
Reactivation 
Uninstallation 
Disengagement 
Account" 
creation 
Virality 
Downloads," 
Gross revenue 
ARPU 
Activation 
Churn, CLV 
In-app" 
purchases 
Legitimate 
Incentivized 
Fraudulent 
Ratings!
Building message maps
Build a message map. 
1. Understand the stages a buyer goes through 2. Create benefits; mitigate objections 3. Target the message to the stage the audience is at
Everyone in the world 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic
Everyone in the world 
A. I need a car 
People who want to drive 
I should buy 
B. a car 
Prospective car buyers 
It should be 
C. a hybrid 
People looking for a hybrid 
I should buy 
D. a Honda Civic 
Honda Civic Hybrid owners
“Isn’t it time you got out of the 
city?” campaign showing how cars 
make nature accessible & ridiculing 
urban hipsters. 
Ads showing how cars are needed 
any time (pregnancy, errands, urgent 
business) and how a car is a 
“personal assistant.” 
Urgency (“every time you drive a 
non-hybrid car you kill the planet a 
little”) and testimonials from buyers 
who’ve saved money. 
Honda branding ads and model-specific 
promotions. 
Follow-up satisfaction campaign to 
encourage buyers to tell their friends 
Everyone in the world 
A. I need a car 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
I should buy 
B. a car 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
It should be 
C. a hybrid 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
I should buy 
D. a Honda Civic 
Honda Civic Hybrid owners
Everyone in the world 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
Honda Civic Hybrid owners 
Those who don’t need cars 
• I’m too young to drive 
• I’m too old to drive 
• I can walk or take public 
transit 
Car users who won’t buy 
• It’s too expensive for me 
• I will use a shared car service 
• It’ll get stolen 
Those who won’t buy hybrids 
• Hybrids are gutless 
• Batteries are toxic & explosive 
• In the end it costs more than 
it saves 
I will buy another brand 
• I buy domestic 
• I’ve always driven a VW 
• Toyotas are reliable 
• I want something prestigious 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic
Everyone in the world 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
Honda Civic Hybrid owners 
Those who don’t need cars 
• I’m too young to drive 
• I’m too old to drive 
• I can walk or take public 
transit 
Car users who won’t buy 
• It’s too expensive for me 
• I will use a shared car service 
• It’ll get stolen 
Those who won’t buy hybrids 
• Hybrids are gutless 
• Batteries are toxic & explosive 
• In the end it costs more than 
it saves 
I will buy another brand 
• I buy domestic 
• I’ve always driven a VW 
• Toyotas are reliable 
• I want something prestigious 
Sponsor a driving school 
“Give the gift of driving” 
campaign for grandparents. 
PR on dangers of commuting, 
pedestrian deaths 
Financing, cashback 
Sell to carshares; 
underscore their limitations 
Theft warranty, tracking 
services, high-end locks 
Independent tests, 
standard metrics (0-60 in X) 
Lab research, studies 
ROI calculator; 
replacement programs 
Prove Honda hires US workers 
“Time to leave Germany” ads 
Spontaneous accel. stories 
Premium brand (Acura) 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic
“Isn’t it time you got out of the 
city?” campaign showing how cars 
make nature accessible & ridiculing 
urban hipsters. 
Ads showing how cars are needed 
any time (pregnancy, errands, urgent 
business) and how a car is a 
“personal assistant.” 
Urgency (“every time you drive a 
non-hybrid car you kill the planet a 
little”) and testimonials from buyers 
who’ve saved money. 
Honda branding ads and model-specific 
promotions. 
Follow-up satisfaction campaign to 
encourage buyers to tell their friends 
Everyone in the world 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
Honda Civic Hybrid owners 
Those who don’t need cars 
• I’m too young to drive 
• I’m too old to drive 
• I can walk or take public 
transit 
Car users who won’t buy 
• It’s too expensive for me 
• I will use a shared car service 
• It’ll get stolen 
Those who won’t buy hybrids 
• Hybrids are gutless 
• Batteries are toxic & explosive 
• In the end it costs more than 
it saves 
I will buy another brand 
• I buy domestic 
• I’ve always driven a VW 
• Toyotas are reliable 
• I want something prestigious 
Sponsor a driving school 
“Give the gift of driving” 
campaign for grandparents. 
PR on dangers of commuting, 
pedestrian deaths 
Financing, cashback 
Sell to carshares; 
underscore their limitations 
Theft warranty, tracking 
services, high-end locks 
Independent tests, 
standard metrics (0-60 in X) 
Lab research, studies 
ROI calculator; 
replacement programs 
Prove Honda hires US workers 
“Time to leave Germany” ads 
Spontaneous accel. stories 
Premium brand (Acura) 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic

Weitere ähnliche Inhalte

Was ist angesagt?

Lean Analytics: A short summary
Lean Analytics: A short summaryLean Analytics: A short summary
Lean Analytics: A short summaryJan König
 
UX design & redesign
UX design & redesignUX design & redesign
UX design & redesignShanshan Ma
 
Lean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean Analytics
 
From Zero to a Million Users - Dropbox and Xobni lessons learned
From Zero to a Million Users - Dropbox and Xobni lessons learnedFrom Zero to a Million Users - Dropbox and Xobni lessons learned
From Zero to a Million Users - Dropbox and Xobni lessons learnedAdam Smith
 
Love the Problem, Not Your Solution
Love the Problem, Not Your SolutionLove the Problem, Not Your Solution
Love the Problem, Not Your SolutionAsh Maurya
 
Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics
 
Test for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing PagesTest for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing PagesOptimizely
 
Measuring UX
Measuring UXMeasuring UX
Measuring UXFITC
 
Think Like A Growth Hacker
Think Like A Growth HackerThink Like A Growth Hacker
Think Like A Growth HackerTim Homuth
 
Lean startup notes
Lean startup notesLean startup notes
Lean startup notesMichael Vax
 
Metrics for Startup Success and Failure
Metrics for Startup Success and FailureMetrics for Startup Success and Failure
Metrics for Startup Success and FailureHiten Shah
 
Growth engine for saas startups
Growth engine for saas startupsGrowth engine for saas startups
Growth engine for saas startupsAlin Dobra
 
Ab testing 101
Ab testing 101Ab testing 101
Ab testing 101Ashish Dua
 
Minimum Viable Product
Minimum Viable ProductMinimum Viable Product
Minimum Viable ProductEric Ries
 
Mobile Growth: Best Strategies, Tools and Tactics
Mobile Growth: Best Strategies, Tools and TacticsMobile Growth: Best Strategies, Tools and Tactics
Mobile Growth: Best Strategies, Tools and TacticsAdrien Montcoudiol
 
How to Accurately Track Marketing Campaigns + Free Campaign Tagging Tool
How to Accurately Track Marketing Campaigns + Free Campaign Tagging ToolHow to Accurately Track Marketing Campaigns + Free Campaign Tagging Tool
How to Accurately Track Marketing Campaigns + Free Campaign Tagging ToolIn Marketing We Trust
 

Was ist angesagt? (20)

Lean Analytics: A short summary
Lean Analytics: A short summaryLean Analytics: A short summary
Lean Analytics: A short summary
 
UX design & redesign
UX design & redesignUX design & redesign
UX design & redesign
 
Lean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basics
 
From Zero to a Million Users - Dropbox and Xobni lessons learned
From Zero to a Million Users - Dropbox and Xobni lessons learnedFrom Zero to a Million Users - Dropbox and Xobni lessons learned
From Zero to a Million Users - Dropbox and Xobni lessons learned
 
Love the Problem, Not Your Solution
Love the Problem, Not Your SolutionLove the Problem, Not Your Solution
Love the Problem, Not Your Solution
 
Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and Enterprises
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
 
Test for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing PagesTest for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing Pages
 
Measuring UX
Measuring UXMeasuring UX
Measuring UX
 
A/B testing
A/B testingA/B testing
A/B testing
 
Think Like A Growth Hacker
Think Like A Growth HackerThink Like A Growth Hacker
Think Like A Growth Hacker
 
Lean startup notes
Lean startup notesLean startup notes
Lean startup notes
 
Startup Metrics
Startup MetricsStartup Metrics
Startup Metrics
 
Metrics for Startup Success and Failure
Metrics for Startup Success and FailureMetrics for Startup Success and Failure
Metrics for Startup Success and Failure
 
Growth engine for saas startups
Growth engine for saas startupsGrowth engine for saas startups
Growth engine for saas startups
 
Ab testing 101
Ab testing 101Ab testing 101
Ab testing 101
 
Minimum Viable Product
Minimum Viable ProductMinimum Viable Product
Minimum Viable Product
 
Growth Hacking
Growth HackingGrowth Hacking
Growth Hacking
 
Mobile Growth: Best Strategies, Tools and Tactics
Mobile Growth: Best Strategies, Tools and TacticsMobile Growth: Best Strategies, Tools and Tactics
Mobile Growth: Best Strategies, Tools and Tactics
 
How to Accurately Track Marketing Campaigns + Free Campaign Tagging Tool
How to Accurately Track Marketing Campaigns + Free Campaign Tagging ToolHow to Accurately Track Marketing Campaigns + Free Campaign Tagging Tool
How to Accurately Track Marketing Campaigns + Free Campaign Tagging Tool
 

Andere mochten auch

Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19Lean Analytics
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Lean Analytics
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsLean Analytics
 
About Mro Facilities
About Mro FacilitiesAbout Mro Facilities
About Mro Facilitieseidderf91
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014Lean Analytics
 
The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ...
 The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ... The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ...
The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ...Traction Conf
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Lean Analytics
 
Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo
Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo
Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo Traction Conf
 
The Holy Grail of Traction - Brian Balfour, HubSpot
The Holy Grail of Traction - Brian Balfour, HubSpotThe Holy Grail of Traction - Brian Balfour, HubSpot
The Holy Grail of Traction - Brian Balfour, HubSpotTraction Conf
 
Facebook's Growth Hacker on how they put Facebook on the Path to 1 Billion Users
Facebook's Growth Hacker on how they put Facebook on the Path to 1 Billion UsersFacebook's Growth Hacker on how they put Facebook on the Path to 1 Billion Users
Facebook's Growth Hacker on how they put Facebook on the Path to 1 Billion Usersgrowthhackersconference
 
State of the Word 2016
State of the Word 2016State of the Word 2016
State of the Word 2016photomatt
 
Setting and Achieving Growth Goals
Setting and Achieving Growth GoalsSetting and Achieving Growth Goals
Setting and Achieving Growth GoalsSean Ellis
 
Building a Company-Wide Growth Culture: SaaStr Annual 2016
Building a Company-Wide Growth Culture: SaaStr Annual 2016Building a Company-Wide Growth Culture: SaaStr Annual 2016
Building a Company-Wide Growth Culture: SaaStr Annual 2016Sean Ellis
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean Analytics
 

Andere mochten auch (17)

Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
 
About Mro Facilities
About Mro FacilitiesAbout Mro Facilities
About Mro Facilities
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for America
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014
 
The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ...
 The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ... The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ...
The Secret to Atlassian’s 10-Year Growth Streak by Archana Agrawal, Head of ...
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo
Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo
Building a Data Driven Growth Organization by Heather Zynczak, CMO, Domo
 
The Holy Grail of Traction - Brian Balfour, HubSpot
The Holy Grail of Traction - Brian Balfour, HubSpotThe Holy Grail of Traction - Brian Balfour, HubSpot
The Holy Grail of Traction - Brian Balfour, HubSpot
 
Facebook's Growth Hacker on how they put Facebook on the Path to 1 Billion Users
Facebook's Growth Hacker on how they put Facebook on the Path to 1 Billion UsersFacebook's Growth Hacker on how they put Facebook on the Path to 1 Billion Users
Facebook's Growth Hacker on how they put Facebook on the Path to 1 Billion Users
 
State of the Word 2016
State of the Word 2016State of the Word 2016
State of the Word 2016
 
Setting and Achieving Growth Goals
Setting and Achieving Growth GoalsSetting and Achieving Growth Goals
Setting and Achieving Growth Goals
 
Building a Company-Wide Growth Culture: SaaStr Annual 2016
Building a Company-Wide Growth Culture: SaaStr Annual 2016Building a Company-Wide Growth Culture: SaaStr Annual 2016
Building a Company-Wide Growth Culture: SaaStr Annual 2016
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014
 

Ähnlich wie Slides from Growthcon 2014 Lean Analytics masterclass

Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Startup Co.
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Lean Startup Co.
 
online startups metrics
online startups metrics online startups metrics
online startups metrics Hatem Kameli
 
SBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsMeasureWorks
 
Help Employees Socialize Your Brand
Help Employees Socialize Your BrandHelp Employees Socialize Your Brand
Help Employees Socialize Your BrandThe Safdar Group
 
Find the Metrics That Matter for Your Newsroom
Find the Metrics That Matter for Your NewsroomFind the Metrics That Matter for Your Newsroom
Find the Metrics That Matter for Your NewsroomOnline News Association
 
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Kate O'Neill
 
Big data can be used at SME's too
Big data can be used at SME's tooBig data can be used at SME's too
Big data can be used at SME's tooGeorge Antony
 
MVP Types, Tools and Social Impact
MVP Types, Tools and Social ImpactMVP Types, Tools and Social Impact
MVP Types, Tools and Social ImpactPaul Orlando
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
 
Introduction to lean analytics
Introduction to lean analyticsIntroduction to lean analytics
Introduction to lean analyticsKartik Narayanan
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking jeroentjepkema
 
101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon Paris101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon ParisDvir Reznik
 
LTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfLTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfjeroen_tjepkema
 
What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)Dave LaFontaine
 
Drive Profitability In Your Organisation with Data and Social Strategies
Drive Profitability In Your Organisation with Data and Social StrategiesDrive Profitability In Your Organisation with Data and Social Strategies
Drive Profitability In Your Organisation with Data and Social Strategies Symphony3
 
Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16
Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16
Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16Kimberly Gomez
 

Ähnlich wie Slides from Growthcon 2014 Lean Analytics masterclass (20)

Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair Croll
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
 
online startups metrics
online startups metrics online startups metrics
online startups metrics
 
SBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean Analytics
 
Practical Social Analytics
Practical Social AnalyticsPractical Social Analytics
Practical Social Analytics
 
Help Employees Socialize Your Brand
Help Employees Socialize Your BrandHelp Employees Socialize Your Brand
Help Employees Socialize Your Brand
 
Find the Metrics That Matter for Your Newsroom
Find the Metrics That Matter for Your NewsroomFind the Metrics That Matter for Your Newsroom
Find the Metrics That Matter for Your Newsroom
 
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
 
Big data can be used at SME's too
Big data can be used at SME's tooBig data can be used at SME's too
Big data can be used at SME's too
 
MVP Types, Tools and Social Impact
MVP Types, Tools and Social ImpactMVP Types, Tools and Social Impact
MVP Types, Tools and Social Impact
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
 
Introduction to lean analytics
Introduction to lean analyticsIntroduction to lean analytics
Introduction to lean analytics
 
Brandable newsletter for printers and mailers
Brandable newsletter for printers and mailersBrandable newsletter for printers and mailers
Brandable newsletter for printers and mailers
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
 
101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon Paris101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon Paris
 
LTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfLTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdf
 
What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)
 
Drive Profitability In Your Organisation with Data and Social Strategies
Drive Profitability In Your Organisation with Data and Social StrategiesDrive Profitability In Your Organisation with Data and Social Strategies
Drive Profitability In Your Organisation with Data and Social Strategies
 
Startup marketing 101
Startup marketing 101Startup marketing 101
Startup marketing 101
 
Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16
Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16
Size Of Writing Paper. Writing Paper Sizes Chart. 2019-01-16
 

Kürzlich hochgeladen

Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...
Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...
Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...Nagarjuna Reddy Aturi
 
Leveraging Gap Analysis for Continuous Improvement
Leveraging Gap Analysis for Continuous ImprovementLeveraging Gap Analysis for Continuous Improvement
Leveraging Gap Analysis for Continuous ImprovementCIToolkit
 
Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...
Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...
Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...CIToolkit
 
HOTEL MANAGEMENT SYSTEM PPT PRESENTATION
HOTEL MANAGEMENT SYSTEM PPT PRESENTATIONHOTEL MANAGEMENT SYSTEM PPT PRESENTATION
HOTEL MANAGEMENT SYSTEM PPT PRESENTATIONsivani14565220
 
Management 11th Edition - Chapter 13 - Managing Teams
Management 11th Edition - Chapter 13 - Managing TeamsManagement 11th Edition - Chapter 13 - Managing Teams
Management 11th Edition - Chapter 13 - Managing Teamsshakkardaddy
 
Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...
Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...
Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...dsnow9802
 
How Technologies will change the relationship with Human Resources
How Technologies will change the relationship with Human ResourcesHow Technologies will change the relationship with Human Resources
How Technologies will change the relationship with Human ResourcesMassimo Canducci
 
BoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & Engineering
BoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & EngineeringBoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & Engineering
BoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & EngineeringBusiness of Software Conference
 
Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...
Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...
Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...Conny Dethloff
 
Operations Management -- Sustainability and Supply Chain Management.pdf
Operations Management -- Sustainability and Supply Chain Management.pdfOperations Management -- Sustainability and Supply Chain Management.pdf
Operations Management -- Sustainability and Supply Chain Management.pdfcoolsnoopy1
 
Management 11th Edition - Chapter 9 - Strategic Management
Management 11th Edition - Chapter 9 - Strategic ManagementManagement 11th Edition - Chapter 9 - Strategic Management
Management 11th Edition - Chapter 9 - Strategic Managementshakkardaddy
 
The Role of Box Plots in Comparing Multiple Data Sets
The Role of Box Plots in Comparing Multiple Data SetsThe Role of Box Plots in Comparing Multiple Data Sets
The Role of Box Plots in Comparing Multiple Data SetsCIToolkit
 
From Red to Green: Enhancing Decision-Making with Traffic Light Assessment
From Red to Green: Enhancing Decision-Making with Traffic Light AssessmentFrom Red to Green: Enhancing Decision-Making with Traffic Light Assessment
From Red to Green: Enhancing Decision-Making with Traffic Light AssessmentCIToolkit
 
Mind Mapping: A Visual Approach to Organize Ideas and Thoughts
Mind Mapping: A Visual Approach to Organize Ideas and ThoughtsMind Mapping: A Visual Approach to Organize Ideas and Thoughts
Mind Mapping: A Visual Approach to Organize Ideas and ThoughtsCIToolkit
 
Digital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic Traits
Digital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic TraitsDigital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic Traits
Digital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic TraitsHannah Smith
 
Exploring Variable Relationships with Scatter Diagram Analysis
Exploring Variable Relationships with Scatter Diagram AnalysisExploring Variable Relationships with Scatter Diagram Analysis
Exploring Variable Relationships with Scatter Diagram AnalysisCIToolkit
 
The Final Activity in Project Management
The Final Activity in Project ManagementThe Final Activity in Project Management
The Final Activity in Project ManagementCIToolkit
 
Management 11th Edition - Chapter 11 - Adaptive Organizational Design
Management 11th Edition - Chapter 11 - Adaptive Organizational DesignManagement 11th Edition - Chapter 11 - Adaptive Organizational Design
Management 11th Edition - Chapter 11 - Adaptive Organizational Designshakkardaddy
 
Adapting to Change: Using PEST Analysis for Better Decision-Making
Adapting to Change: Using PEST Analysis for Better Decision-MakingAdapting to Change: Using PEST Analysis for Better Decision-Making
Adapting to Change: Using PEST Analysis for Better Decision-MakingCIToolkit
 
The Role of Histograms in Exploring Data Insights
The Role of Histograms in Exploring Data InsightsThe Role of Histograms in Exploring Data Insights
The Role of Histograms in Exploring Data InsightsCIToolkit
 

Kürzlich hochgeladen (20)

Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...
Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...
Advancing Enterprise Risk Management Practices- A Strategic Framework by Naga...
 
Leveraging Gap Analysis for Continuous Improvement
Leveraging Gap Analysis for Continuous ImprovementLeveraging Gap Analysis for Continuous Improvement
Leveraging Gap Analysis for Continuous Improvement
 
Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...
Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...
Paired Comparison Analysis: A Practical Tool for Evaluating Options and Prior...
 
HOTEL MANAGEMENT SYSTEM PPT PRESENTATION
HOTEL MANAGEMENT SYSTEM PPT PRESENTATIONHOTEL MANAGEMENT SYSTEM PPT PRESENTATION
HOTEL MANAGEMENT SYSTEM PPT PRESENTATION
 
Management 11th Edition - Chapter 13 - Managing Teams
Management 11th Edition - Chapter 13 - Managing TeamsManagement 11th Edition - Chapter 13 - Managing Teams
Management 11th Edition - Chapter 13 - Managing Teams
 
Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...
Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...
Hajra Karrim: Transformative Leadership Driving Innovation and Efficiency in ...
 
How Technologies will change the relationship with Human Resources
How Technologies will change the relationship with Human ResourcesHow Technologies will change the relationship with Human Resources
How Technologies will change the relationship with Human Resources
 
BoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & Engineering
BoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & EngineeringBoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & Engineering
BoSUSA23 | Chris Spiek & Justin Dickow | Autobooks Product & Engineering
 
Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...
Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...
Better SAFe than sorry - Why scaled agile frameworks do not necessarily impro...
 
Operations Management -- Sustainability and Supply Chain Management.pdf
Operations Management -- Sustainability and Supply Chain Management.pdfOperations Management -- Sustainability and Supply Chain Management.pdf
Operations Management -- Sustainability and Supply Chain Management.pdf
 
Management 11th Edition - Chapter 9 - Strategic Management
Management 11th Edition - Chapter 9 - Strategic ManagementManagement 11th Edition - Chapter 9 - Strategic Management
Management 11th Edition - Chapter 9 - Strategic Management
 
The Role of Box Plots in Comparing Multiple Data Sets
The Role of Box Plots in Comparing Multiple Data SetsThe Role of Box Plots in Comparing Multiple Data Sets
The Role of Box Plots in Comparing Multiple Data Sets
 
From Red to Green: Enhancing Decision-Making with Traffic Light Assessment
From Red to Green: Enhancing Decision-Making with Traffic Light AssessmentFrom Red to Green: Enhancing Decision-Making with Traffic Light Assessment
From Red to Green: Enhancing Decision-Making with Traffic Light Assessment
 
Mind Mapping: A Visual Approach to Organize Ideas and Thoughts
Mind Mapping: A Visual Approach to Organize Ideas and ThoughtsMind Mapping: A Visual Approach to Organize Ideas and Thoughts
Mind Mapping: A Visual Approach to Organize Ideas and Thoughts
 
Digital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic Traits
Digital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic TraitsDigital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic Traits
Digital PR Summit - Leadership Lessons: Myths, Mistakes, & Toxic Traits
 
Exploring Variable Relationships with Scatter Diagram Analysis
Exploring Variable Relationships with Scatter Diagram AnalysisExploring Variable Relationships with Scatter Diagram Analysis
Exploring Variable Relationships with Scatter Diagram Analysis
 
The Final Activity in Project Management
The Final Activity in Project ManagementThe Final Activity in Project Management
The Final Activity in Project Management
 
Management 11th Edition - Chapter 11 - Adaptive Organizational Design
Management 11th Edition - Chapter 11 - Adaptive Organizational DesignManagement 11th Edition - Chapter 11 - Adaptive Organizational Design
Management 11th Edition - Chapter 11 - Adaptive Organizational Design
 
Adapting to Change: Using PEST Analysis for Better Decision-Making
Adapting to Change: Using PEST Analysis for Better Decision-MakingAdapting to Change: Using PEST Analysis for Better Decision-Making
Adapting to Change: Using PEST Analysis for Better Decision-Making
 
The Role of Histograms in Exploring Data Insights
The Role of Histograms in Exploring Data InsightsThe Role of Histograms in Exploring Data Insights
The Role of Histograms in Exploring Data Insights
 

Slides from Growthcon 2014 Lean Analytics masterclass

  • 1. Practical Lean Analytics Growthcon October 24, 2014 @acroll
  • 2. Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
  • 3. The core of Lean is iteration.
  • 4. Most startups don’t know what they’ll be when they grow up. Hotmail was a database company Flickr was going to be an MMO Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
  • 5. Waterfall, agile, and lean (Why the old ways don’t work.)
  • 6. Waterfall approach You know the problem and the solution.
  • 7.
  • 8. Known set of requirements Known ways to satisfy them Spec Build Test Launch
  • 9. Agile methodologies Know the problem, find the solution
  • 10.
  • 11. Known set of requirements Unclear how to satisfy them Problem Build Test Viable? Launch statement Sprints Adjust Unknown set of
  • 12. Lean approach First, know that you don’t know.
  • 13. Product/market hypothesis Trial startup Possible problem space Product/ market hypothesis Trial startup Product/ market hypothesis Trial startup Trial startup Product/market hypothesis You are here PIVOT
  • 14. Why now? First: High rate of change of digital technologies & channels.
  • 16. Times a song in “heavy rotation” is played daily 30 15 0 6 26 2007 2012
  • 17. Why now? Second: It’s no longer about whether you can build it—it’s about whether anyone will care.
  • 18. The Attention Economy “What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” (Computers, Communications and the Public Interest, pages 40-41, Herbert Simon Martin Greenberger, ed., The Johns Hopkins Press, 1971.)
  • 19. Lit motors tests the risky part
  • 20. Unfortunately, it is hard to be honest with ourselves.
  • 21. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) No data, no learning. This is where things fall apart.
  • 23. Analytics is the measurement of movement towards your business goals.
  • 24. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 26. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing What will you do differently based on the results you collect?
  • 27. The simplest rule If a metric won’t change how you behave, it’s a bad metric. h"p://www.flickr.com/photos/circasassy/7858155676/
  • 28. Metrics help you know yourself. Acquisition Hybrid Loyalty You are just like 70% of retailers 20% of retailers 10% of retailers Customers that buy >1x in 90d Your customers will buy from you Once 2-2.5 per year >2.5 per year Then you are in this mode 1-15% 15-30% >30% Focus on Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)
  • 29. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  • 30. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  • 31. Rumsfeld on Analytics Things we know don’t know (Or rather, Avinash Kaushik channeling Rumsfeld) we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
  • 32. Slicing and dicing data Feb Mar Apr May 5,000 Active users 0 Jan Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis Changing several things at once to see which correlates with a result. ☀☁☀☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  • 33. Which of these two companies is doing better?
  • 34. January February March April May Is this company Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 How about this one?
  • 35. Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 36. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 37. Some examples A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 38. Which means it’s time to talk about correlation.
  • 39. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption Drownings
  • 40. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 41. A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 42. Why is Nigerian spam so badly written?
  • 43. Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” This would be horribly inefficient since humans are involved. Good language (10% conversion) Not-gullible (.07% conversion) Aunshul Rege of Rutgers University, USA in 2009 1000 emails Bad language (0.1% conversion) 1-2 responses Gullible (70% conversion) 1 fool and their money, parted. 1000 emails 100 responses 1 fool and their money, parted.
  • 44. Turns out the word “Nigeria” is the best way to identify promising prospects.
  • 45. Nigerian spammers really understand their target market. They see past vanity metrics.
  • 46. The Lean Analytics framework.
  • 47. Sustainable growth comes through the actions of your customers. - Eric Ries
  • 48. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  • 49. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 50. Gate Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. The five stages
  • 51. Empathy stage: Localmind hacks Twitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
  • 52. LikeBright’s mechanical turk Used Mechanical Turk, Google Voice to speak w/ 100 single women; paid $2. The interviews lasted typically around 10-15 minutes. Simple interview script with open-ended questions, since he was digging into the problem validation stage of his startup. Founder Nick Soman: “I was amazed at the feedback I got. We were able to speak with one hundred single women that met our criteria in four hours on one evening.” Went back to TechStars and got accepted. LikeBright’s website is now live with a 50% female user base, and recently raised a round of funding. “Since that first foray into interviewing customers, I’ve probably spoken with over a thousand people through Mechanical Turk,”
  • 53. How to avoid leading the witness Avoid biased wording, preconceptions, or a giveaway appearance. Word your surveys carefully to be neutral. Get them to purchase. Ask them to pay. Demand real Ask “why” several times. Leave lingering, uncomfortable pauses Don’t tip your hand Make the question real Keep digging Look for other clues Have a colleague make notes of when they react, or of their body
  • 54. Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient installs mobile app Recipient creates account, profile Recipient can edit profile, etc. Recipient reads survey question Recipient responds to question Recipient sees survey results 10-25% RESPONSE RATE 70-90% RESPONSE RATE
  • 55. 1200 1000 800 600 400 200 0 January February 1 2 3 4 5 6 7 8 9 Days since last engagement 25000 20000 15000 10000 5000 0 Disengaged (>10 days) Number of users A better approach to engagement This is a good thing.
  • 56. Who is worth more? A Lifetime: B Lifetime: Today $200 $200 Roberto Medri, Etsy Visits
  • 57. Virality stage: Timehop focuses on content sharing Focused on percent of daily active users that share their content Aiming for 20-30% of DAU sharing “All that matters now is virality. Everything else—be it press, publicity stunts or something else—is like pushing a rock up a mountain: it will never scale. But being viral will.” - Jonathan Wegener, co-founder
  • 58. ------------------------------------------------------ Get your free private email at http://www.hotmail.com ------------------------------------------------------
  • 59.
  • 60. v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0 http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ Viral coefficient
  • 61. Or simpler x - > 1 Users Viral coefficient Churn & abandonment
  • 62. Revenue stage: Backupify’s Customer Acquisition Payback Initially focused on site visitors Then focused on trials Then switched to signups Today, MRR In early 2010, CAC was $243 and ARPU was only $39 Pivoted to target business users CLV-to-CAC today is 5-6x Now they track Customer Acquisition Payback Target is less than 12 months
  • 63. Scale stage: Incremental order cost Marginal costs Fixed costs
  • 64. Six business model archetypes. E-commerce SaaS Mobile Media app User-gen content 2-sided market The business you’re in
  • 65. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 66. Acquisition Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 67. Activation Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 68. Retention Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 69. Revenue Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 70. Revenue Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 71. Model + Stage = One Metric That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG One Metric That Matters. Empathy Stickiness Virality Revenue Scale The stage you’re at
  • 74. In a startup, focus is hard to achieve.
  • 75. Having only one metric addresses this problem.
  • 77. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  • 78. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 79. Empathy Stickiness Virality Revenue Scale E-commerce Mobile app User-gen content SaaS Media 2-sided market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation (Money from transactions) Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC (Money from active users) Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing (Money from ad clicks) Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses
  • 81. Drawing some lines in the sand.
  • 82. A company loses a quarter of its customers every year. Is this good or bad?
  • 83. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 84. Not knowing what normal is makes you do unwise things.
  • 85. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  • 86. Baseline: 10% visitor engagement/day 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1%of users/day use it concurrently Fred Wilson’s social ratios
  • 87. Baseline: Calculating customer lifetime 25% 5% monthly churn monthly churn 100/25=4 100/5=20 The average The average customer lasts customer lasts 4 months 20 months 2% monthly churn 100/2=50 The average customer lasts 50 months
  • 88. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV 1/3 spend $200 CAC Now segment those users!
  • 89. Etsy • Online store for creative types, founded 2005 • $525M Gross Merchandise Sales in 2011, with 19,000,000 members and 800,000 active shops offering 15,000,000 items for sale • 1.4B pageviews per month ~2M iPhone app downloads • Thin revenues: Etsy makes only $0.20 or 3.5% margin • Heavy focus on Customer Lifetime Value (buyer and seller) • Actually residual lifetime value; they take this pretty seriously.
  • 90. Etsy • The best customers to target are • Recent high-profile customers • Old-time best customers about to churn or just churned • Tiered campaigns • Bronze/silver customers: reinforcement, nudges • Gold customers: premium services • Platinum customers: recognition • What they watch: • Growth of individual product categories • Time to first sale by a user • Average order value • Percentage of visits that convert to a sale • Percentage of return buyers • Distinct sellers within a product category • Time-to-first-sale and average order value by product category Roberto Medri, Etsy
  • 92. Pick a KPI Draw a line Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
  • 93. Do AirBnB hosts get more business if their property is professionally photographed?
  • 94. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 95. 5,000 shoots per month by February 2012
  • 96. Hang on a second.
  • 97. REALLY? Gut instinct (hypothesis) Professional photography helps AirBnB’s business
  • 98. Pick a KPI Draw a line Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
  • 99. “Gee, those houses that do well look really nice.” Maybe it’s the camera. With data: find a commonality “Computer: What do all the highly rented houses have in common?” Camera model. Without data: make a good guess
  • 100. Circle of Moms: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
  • 101. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User analytics Spying on users User interaction Customer User segmentation satisfaction KPI dashboards
  • 102. Growth hacking (is a word you should hate but will hear a lot about.)
  • 103. Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
  • 106. Guerrilla marketing Data-driven learning GROWTH HACKING Subversiveness
  • 108.
  • 109.
  • 110.
  • 114. Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
  • 117. 1. Define your business model
  • 118. 2. Draw a system diagram
  • 119. 3. Decide what stage you’re at
  • 120. 4. Identify the One Metric That Matters (usually the one that is most broken)
  • 121. 5. Use the cycle to experiment until you’ve achieved the desired result.
  • 122. 6. Set up monitoring for this metric in case it breaks, and choose a new OMTM
  • 124. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 125. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 126. ARCHIMEDES HAD TAKEN BATHS BEFORE.
  • 127. Once, a leader convinced others in the absence of data.
  • 128. Now, a leader knows what questions to ask.
  • 129. Ben Yoskovitz byosko@gmail.com @byosko Alistair Croll acroll@gmail.com @acroll
  • 130.
  • 131.
  • 132. The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases App store! App sales Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU Activation Churn, CLV In-app" purchases Legitimate Incentivized Fraudulent Ratings!
  • 133.
  • 134.
  • 135.
  • 137. Build a message map. 1. Understand the stages a buyer goes through 2. Create benefits; mitigate objections 3. Target the message to the stage the audience is at
  • 138. Everyone in the world A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
  • 139. Everyone in the world A. I need a car People who want to drive I should buy B. a car Prospective car buyers It should be C. a hybrid People looking for a hybrid I should buy D. a Honda Civic Honda Civic Hybrid owners
  • 140. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model-specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends Everyone in the world A. I need a car People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” I should buy B. a car Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” It should be C. a hybrid People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” I should buy D. a Honda Civic Honda Civic Hybrid owners
  • 141. Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
  • 142. Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. PR on dangers of commuting, pedestrian deaths Financing, cashback Sell to carshares; underscore their limitations Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
  • 143. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model-specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. PR on dangers of commuting, pedestrian deaths Financing, cashback Sell to carshares; underscore their limitations Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic