SlideShare ist ein Scribd-Unternehmen logo
1 von 328
Downloaden Sie, um offline zu lesen
Lean Analytics
New Media Manitoba
June, 2015
@acroll
I can’t tell you how to succeed
(If I could, I’d be starting your company instead of standing here.)
I can tell you how to avoid failure
(At least, some forms of it.)
Percent of businesses that fail
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10
71%69%66%63%60%55%50%44%36%25%
http://www.statisticbrain.com/startup-failure-by-industry/
Still operating after 4 years
Finance Insurance and Real Estate
Education and Health
Agriculture
Services
Wholesale
Mining
Manufacturing
Construction
Retail
Transportation Communication and Utilities
Information 37%
45%
47%
47%
49%
51%
54%
55%
56%
56%
58%
http://www.statisticbrain.com/startup-failure-by-industry/
Why firms die
http://www.statisticbrain.com/startup-failure-by-industry/
Neglect,

fraud,

disaster

1%
Lack of industry

experience

13%
Lack of managerial

experience

34%
Incompetence

52%
Emotional Pricing
Living too high for the business
Nonpayment of taxes
No knowledge of pricing
Lack of planning
No knowledge of financing
No experience in record-keeping
Poor credit granting
practices
Expansion too rapid
Inadequate borrowing
practices
Carry inadequate inventory
No knowledge of suppliers
Wasted advertising budget
Part one:

Introduction to Lean Analytics
Don’t sell what you can make. Make what you can sell.
Kevin Costner is a lousy entrepreneur.
The core of Lean
is iteration.
Waterfall approach
You know the problem and the solution.
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Known set of
requirements
Known ways to satisfy
them
Spec Build Test Launch
Agile methodologies
Know the problem, find the solution
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Known set of
requirements
Unclear how to satisfy
them
Build Test LaunchViable?Problem

statement
Adjust
Sprints
Unknown set of
requirements
Lean approach
First, know that you don’t know.
Possible problem
space
Product/
market
hypothesis
Trial startup
Product/
market
hypothesis
Trial startup
Product/market
hypothesis
Trialstartup
Product/market
hypothesis
Trialstartup
You are
herePIVOT
Unfortunately,
we’re all liars.
Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
No data, no
learning.
Most startups don’t know what they’ll
be when they grow up.
Hotmail

was a
database
company
Flickr

was going to
be an MMO
Twitter

was a
podcasting
company
Autodesk

made
desktop
automation
Paypal

first built for
Palmpilots
Freshbooks

was invoicing
for a web
design firm
Wikipedia

was to be
written by
experts only
Mitel

was a
lawnmower
company
Gottfried von Leibniz
was a geek just like you.
First calculator

(stepped reckoner)
One of the first
to recognize the
importance of
binary.
“I thought again
about my early plan
of a new language or
writing-system of
reason, which could
serve as a
communication tool
for all different
nations..”
The best of all possible
worlds is the one in
which the fewest
starting conditions
produce the greatest
variety of outcomes.
Analytics can help.
Analytics is the measurement of
movement towards your business
goals.
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
Why now?
First: High rate of change of digital
technologies & channels.
Arbitron and radio data
Times a song in “heavy
rotation” is played daily
0
15
30
2007 2012
266
Some fundamentals.
Analytics, performance, aggregation, and the right metrics
We’re bad at data.
I have two kids.
At least one of them is a girl.
What are the chances
the other is a boy?
BB BG
GB GG
2 of 3 (66%) are boys.
GB GG BG
Fundamental:

What makes a good metric?
A good metric is:
Understandable
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparative
Comparison is
context.
A ratio or rate
The only way to
measure
change and roll
up the tension
between two
metrics (MPH)
Behavior

changing
What will you
do differently
based on the
results you
collect?
The
simplest
rule
bad

metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%

of retailers
20%

of retailers
10%

of retailers
You are
just like
Customers that
buy >1x in 90d
Once
2-2.5

per year
>2.5

per year
Your customers
will buy from you
Then you are
in this mode
1-15%
15-30%
>30%
Low acquisition
cost, high checkout
Increasing return
rates, market share
Loyalty, selection,
inventory size
Focus on
(Thanks to Kevin Hillstrom for this.)
Qualitative
Unstructured, anecdotal,
revealing, hard to
aggregate, often too
positive & reassuring.
Warm and fuzzy.
Quantitative
Numbers and stats.
Hard facts, less insight,
easier to analyze; often
sour and disappointing.
Cold and hard.
Exploratory
Speculative. Tries to find
unexpected or
interesting insights.
Source of unfair
advantages.
Cool.
Reporting
Predictable. Keeps you
abreast of the normal,
day-to-day operations.
Can be managed by
exception.
Necessary.
Rumsfeld on Analytics
(Or rather, Avinash Kaushik channeling Rumsfeld)
Things we
know
don’t

know
we know Are facts which may be wrong and
should be checked against data.
we don’t

know
Are questions we can answer by
reporting, which we should baseline
& automate.
we know
Are intuition which we should
quantify and teach to improve
effectiveness, efficiency.
we don’t

know
Are exploration which is where
unfair advantage and interesting
epiphanies live.
MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this is the April cohort)
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
Multivariate

analysis
Changing several
things at once to
see which correlates
with a result.
☀
☁
☀
☁
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
☀
☁
Which of these two companies
is doing better?
  January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April   $8 $7
May       $9
How about
this one?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1  
March $7 $6 $5    
April $8 $7      
May $9        
Averages $7 $5 $3 $1 $0.5
Look at the
same data
in cohorts
Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.
Explaining the
past.
Leading
Forward-looking.
Number today that
predicts tomorrow;
reports the news.
Example: pipeline.
Predicting the
future.
Fundamental:

Correlation
1
10
100
1000
10000
Ice cream consumption Drownings
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Correlated
Two variables that are
related (but may be
dependent on
something else.)
Ice cream &
drowning.
Causal
An independent variable
that directly impacts a
dependent one.
Summertime &
drowning.
A leading, causal metric
is a superpower.
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Is social action a leading
indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Is mobile use?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Soundcloud
Cory Levinson, Soundcloud
A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one file in one folder on one device
(ChenLi Wang)
Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
Some examples
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
AirBnB and Craigslist
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
http://www.flickr.com/photos/bootbearwdc/1243690099/
Think subversively.
Why is Nigerian spam so badly
written?
Aunshul Rege of Rutgers University, USA in 2009
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages
emailed; they expect to land 2 or 3 “Mugu” (fools) each week.
One scammer boasted “When you get a reply it’s 70% sure you’ll get the money”
“By sending an email that repels all but the most gullible,” says [Microsoft Researcher
Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts
the true to false positive ratio in his favor.”
1000 emails
1-2 responses
1 fool and their money, parted.
Bad language (0.1% conversion)
Gullible (70% conversion)
1000 emails
100 responses
1 fool and their money, parted.
Good language (10% conversion)
Not-gullible (.07% conversion)
This would be horribly
inefficient since
humans are involved.
Turns out the word “Nigeria” is the best
way to identify promising prospects.
Nigerian spammers
really understand their target market.
They see past vanity metrics.
Fundamental:
KISS
“It can scarcely be denied
that the supreme goal of all theory is
to make the irreducible basic elements
as simple and as few as possible without
having to surrender the adequate
representation of a single datum of
experience.”
http://media.photobucket.com/image/einstein/derekabril/einstein_010.png
“As simple as possible,

but no simpler.”
*(FYI, this is irony.)
The Lean Analytics framework.
Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spend money to
get customers.
Customers are
worth more than
they cost.
Stickiness
Keep people
coming back.
Approach
Get customers
faster than you
lose them.
Math that
matters
Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Retention
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Revenue
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Referral
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
Gate
Thefivestages
Do you have a business model?
http://www.flickr.com/photos/tinfoilraccoon/197640807/
A lemonade stand’s business model is simple.
Yours should be too.
A hobby.
PhotobyRakkaonFlickr.https://www.flickr.com/photos/rakka/6889638004
Market Sizing
Top-down and bottoms-up
Top-down
Start with the whole planet, then narrow it down.
Population of Canada
Percent mobile phones by country
IOS penetration
Put this all together
Metric/Assumption Value
Population of Canada 34.88M
Mobile phone percentage 80%
Mobile phones in Canada 27,904M
IOS percentage 15.5%
IOS phones in Canada 4.32M
IOS devices per account 3.1*
IOS accounts we can sell to in Canada
(this is our Total Addressable Market)
1.39M
* I made this up. Don’t believe everything you read.
How much is the TAM worth?
Metric/Assumption Value
TAM 1.39M
Percent we will claim 5%
Number of users 69,500
User lifetime 40 months
Revenue per month $5 per month
CLV $200
Expected total revenues $13.9M
* I made this up. Don’t believe everything you read.
Bottoms-up
Start with one buyer, then add it up
Bottoms-up
Metric/Assumption Value
Impressions (acquisition) 9,266,667
Percent that install 10%
Installs (activation) 926,667
Percent still using after 30 days 25%
Become regular users (retention) 231,667
Percent that buy 30%
Buy premium version (revenue) 69,500
Percent that churn each month 2.5%
Customer lifespan 40
Revenue per month 5
Customer lifetime value 200
TAM value $13,900,000
Reality check
Bottoms-up (TAM) Value
Impressions (acquisition) 9.3 M impressions
Top-down (TAM) Value
IOS accounts we can sell to in Canada
(this is our Total Addressable Market) 1.39 M users in Canada
You Shall Not Pass.
Six business model archetypes.
E-commerce SaaS Media
Mobile

app
User-gen

content
2-sided

market
The business you’re in
Can you move enough
customers through this
cycle to make more money
than you spend?
Make them
refer others
Retain them
Get revenue from them
Acquire
customers
Activate them
http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
(Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium
churn
Engaged User
Free user
disengagement
Reactivate
Cancel
Trial abandonment
rate
Invite Others
Paying Customer
Reactivation

rate
Paid
conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficient

Viral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling
rate Upselling
Disengaged DissatisfiedTrial Over
Model + Stage = One Metric That Matters.
One Metric

That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric
addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics
SaaS-based SEO toolkit in the scale stage. Focused on net adds.
Was a marketing campaign successful?
Were customer complaints lowered?
Was a product upgrade valuable?
Net adds up:
Can we acquire more valuable customers?
What product features can increase engagement?
Can we improve customer support?
Net adds flat:
Are the new customers not the right segment?
Did a marketing campaign fail?
Did a product upgrade fail somehow?
Is customer support falling apart?
Net adds down:
Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile

app
User-gen

content
2-sided

market
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
CAC, shares,
reactivation
Transaction,
CLV
Affiliates,
white-label
Engagement,
churn
Inherent
virality, CAC
Upselling,
CAC, CLV
API, magic #,
mktplace
Content,
spam
Invites,
sharing
Ads,
donations
Analytics,
user data
Inventory,
listings
SEM, sharing
Transactions,
commission
Other
verticals
(Money from transactions)
Downloads,
churn, virality
WoM, app
ratings, CAC
CLV,
ARPDAU
Spinoffs,
publishers
(Money from active users)
Traffic, visits,
returns
Content
virality, SEM
CPE, affiliate
%, eyeballs
Syndication,
licenses
(Money from ad clicks)
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its
customers every year.
Is this good or bad?
Not knowing what normal is
makes you do stupid things.
Baseline:
5-7% growth a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doing exceptionally well. If you
can only manage 1%, it's a sign
you haven't yet figured out
what you're doing.” At revenue
stage, measure growth in
revenue. Before that, measure
growth in active users.
Paul Graham, Y Combinator
• Are there enough people who really care
enough to sustain a 5% growth rate?

• Don’t strive for a 5% growth at the expense
of really understanding your customers
and building a meaningful solution

• Once you’re a pre-revenue startup at or
near product/market fit, you should have
5% growth of active users each week
• Once you’re generating revenues, they
should grow at 5% a week
It’s oxygen
You need customers to keep learning
It’s a substitute for solvency
PhotobyPaulMilleronFlickr.https://www.flickr.com/photos/94674772@N03/8788576498
Baseline:
10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day use web or mobile app
1% of users/day use it concurrently
Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.
Baseline:
Calculating customer lifetime
25%

monthly churn
100/25=4

The average
customer lasts
4 months
5%

monthly churn
100/5=20

The average
customer lasts
20 months
2%

monthly churn
100/2=50

The average
customer lasts
50 months
Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time to find
out whether your churn and revenue projections
are right
• Cashflow: You’re basically “loaning” the
customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a
CAC of only a third of your CLV will forces you
to verify costs sooner.
Lifetime of 20 mo.
$30/mo. per
customer
$600 CLV
$200 CAC
Now segment
those users!
1/3 spend
Who is worth more?
Today
A
Lifetime:
$200
Roberto Medri, Etsy
B
Lifetime:
$200
Visits
The Lean Analytics cycle
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:

find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
The minimum needed to

test the core idea
Survey owner adds recipient to group
Survey owner asks question
Recipient reads survey question
Recipient responds to question
Recipient sees survey results
(Later, if needed…)
Recipient visits site; no password!
Recipient does password recovery
One-time link sent to email
Recipient creates password
Recipient can edit profile, etc.
Survey owner adds recipient to group
Survey owner asks question
Recipient gets invite
Recipient reads survey question
Recipient responds to question
Recipient installs mobile app
Recipient creates account, profile
Recipient sees survey results
Recipient can edit profile, etc.
10-25%RESPONSERATE
70-90%RESPONSERATE
Do AirBnB hosts
get more business
if their property is
professionally
photographed?
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts
5,000 shoots per month
by February 2012
Hang on a second.
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
SRSLY?
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:

find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
“Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses have in
common?”
Camera model.
With data:

find a commonality
Without data: make a
good guess
Circle of Moms: Not enough engagement
• Too few people were
actually using the
product
• Less than 20% of any
circles had any activity
after their initial creation
• A few million monthly
uniques from 10M
registered users, but no
sustained traction
• They found moms were far more engaged
• Their messages to one another were on average 50% longer
• They were 115% more likely to attach a picture to a post they wrote
• They were 110% more likely to engage in a threaded (i.e. deep)
conversation
• Circle owners’ friends were 50% more likely to engage with the circle
• They were 75% more likely to click on Facebook notifications
• They were 180% more likely to click on Facebook news feed items
• They were 60% more likely to accept invitations to the app
• Pivoted to the new market, including a name change
• By late 2009, 4.5M users and strong engagement
• Sold to Sugar, inc. in early 2012
Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publisher analytics
SaaS analytics
Gaming analytics
User interaction Customer satisfaction KPI dashboardsUser segmentation
User analytics Spying on users
Part two:

Why big companies

need to change
Slides from New Media Manitoba Lean Analytics workshop, June 2015
(http://csinvesting.org/2012/01/06/fortune-500-extinction/)
F500 Life
Expectancy
Growth by entering
a new business 95

% fail
Corporate
Strategy Board
99

% fail
Clay
Christensen
75
years
15
years
1950 2010...
mikemace.com
The slow
death of
a market
leader.
Revenue over time

This is what most
managers track.
Note that sales keep
rising (making you
feel safe) until you
run off the edge of
the cliff.
“Let’s cut
prices to
accelerate
our growth.”
“Time to enter
the mainstream.
Cut prices.”
“We may miss the quarter.
Let’s do a price promotion.”
“That wasn’t
supposed to
happen. We’ll
have to lay
some people
off.
Gross margin
percent

Declining profit
per unit (gross
margin) is actually
your best signal of
trouble.
The adoption curve
Here’s where you
actually are, but you
don’t know it
because you can’t
draw the curve until
after the market
saturates.
Early
adopters
Late
adopters
In other words, if your job is change you
have your work cut out for you.
Lesson one:
Companies die because they fail to
move to new business models.
Clay Christensen, The Innovator’s Dilemma
CostperMB
$1000
$100
$10
$1
Time
14”
M
ainfram
e
8”
M
inicom
puter
5.25”
Desktop
3.5”
N
otebook
Technologies
outstrip what
the market
needs, driven
by feedback
from the
“best”
current
customer.
Clay Christensen, The Innovator’s Dilemma
$1000
$100
$10
$1
Time
8” 5.25”
High end
customer
Low end
customer
The new
market has
different criteria
for success,
which are
uninteresting to
incumbents.
Clay Christensen, The Innovator’s Dilemma
$1000
$100
$10
$1
Time
Storage

capacity
Portability
Sometimes
this has
unintended
consequences
Clay Christensen, The Innovator’s Dilemma
$1000
$100
$10
$1
Time
Smaller disc size means less vibration
impact, leading to greater density,
increasing storage capacity
Why now?
First: High rate of change of digital
technologies & channels.
Arbitron and radio data
Times a song in “heavy
rotation” is played daily
2007 2012
266
This alone explains the collapse of
modern print media.
Circulation, annually Clicks, instantaneously
Letters to the editor, weekly Hashtags, always
Why now?
Second: It’s no longer about whether
you can build it—it’s about whether
anyone will care.
The Attention Economy
“What information consumes
is rather obvious: it consumes
the attention of its recipients.
Hence a wealth of information
creates a poverty of attention, and a
need to allocate that attention efficiently
among the overabundance of
information sources that might
consume it.”
(Computers, Communications and the Public Interest, pages 40-41,
Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon
Lit motors tests the risky part
Lesson two:
The difference between a rogue agent
and a special operative is permission.
If big firms
can’t innovate,
it’s this guy’s
fault.
Technology has radically changed the
incremental cost of businesses.
http://www.flickr.com/photos/ebolasmallpox/3733059220/
Software is eating the world.
An economic order quantity
of one.
Crafted
Mass-
produced
Automated Digital
Quantity Few Many Some One
Cost High Low Medium Free
Lead time Small Large Medium None
Self-service Medium None Some Lots
Customization High None Some Lots
• Cloud computing
• Social media
• 3D printing
• Per-customer
analysis
• Mobile tracking
• Etc...
This is why
software is
eating the
world.
Sustainable competitive advantage allows for
inertia and power to build up along the lines of
an existing business model, which will soon die.
Instead, seek transient competitive
advantage.
Rita Gunther McGrath, The End of Competitive Advantage
Scale is now a liability. Compete on
cycle time.
CW&T make a pen
http://www.flickr.com/photos/art_es_anna/288880795/
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Optimizing the probable means
discounting the possible.
Slides from New Media Manitoba Lean Analytics workshop, June 2015
This isn’t about a lack of resources.
http://www.flickr.com/photos/maladjusted/5207565912
Blockbuster had a lot going for it.
Plenty of inventory, of course.
But that matters less than...
...market intelligence,
customers, existing
payment approval,
and customer
history.
The problem was framing:
Blockbuster thought it was in the video
store management business. Netflix
realized it was in the entertainment
delivery business.
YOU ARE
HERE
YOU ARE
HERE
LOCAL

MAXIMUM
OPTIMIZATION

OF CURRENT
METRICS
YOU ARE
HERE
GLOBAL

MAXIMUMINNOVATION

WITH NEW

RULES
YOU ARE
HERE
SHORT-TERM INVESTORS

HATE GOING DOWNHILL
• $1B invested in Nook
• $475M operating loss
in April 2013
• CEO gone
First mover advantage happens
long before the market emerges.
Constraints slow things down
vs.
Capital cycles don’t fit the short, iterative
nature of startup uncertainty
12 month budgeting cycle; annual plan. Future based on past.
Agile, scrum, lean iterations. Today’s model. No evidence about the future.
Project
Project
Project
Project
Project
Project
Project
Project
Project
Project
Project
Project
(Requires budget insulation)
What’s the
biggest
problem

in a hair
salon?
The empty chair.
Knowing this, how
might you change
the client/customer
relationship?
Can and must
What triggers cultural change?
“Using Cue, you can tell if someone
has the ‘flu in 10 minutes.”
http://tiltthewindmill.com/when-can-becomes-must/
“Using Cue, you must tell if someone
has the ‘flu in 10 minutes…
…or Johnny can’t come on the school trip.”
Part three:
Three maxima of innovation
When you’re a startup
your goal is to find a sustainable,
repeatable business model.
When you’re a big company
your goal is to perpetuate one.
Intrapreneur:
Someone working to produce
disruptive change in an organization
that has already found a sustainable,
repeatable business model.
Business model vs.
company stage
Company size/age
Early stage Big/incumbent
B2B
Target

market
B2C
LessWoM

Moreformaldecisions
Slower cycle time

More legacy constraints
It is way too easy to
mix these up.
Intrapreneurs
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
In a big company,
analytics replaces opinion with fact.
Companies that use data-driven
analytics instead of intuition have
5%-6% higher productivity and
profits than competitors.
Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven
Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).
2011 MIT study of 179 large publicly traded firms
The fundamental shift from Big Data
Ask
question
Define

schema
Collect

data
Answer

question
Refine
problem
Collect
data
Ask
question
Emergent
schema
Explore
data
Answer

question
“Collect first; ask questions later.”
Three kinds of innovation
Sustain/core

(optimizing for more of the same)
Innovate/adjacent

(introduce nearby product,

market, or method)
Disrupt/transformative

(Fundamentally changing

the business model)
Improve along
current metrics...
...or alter

the rate of
improvement
Switch to a new
value model
Change the business

model entirely
Many models for enterprise innovation
Core Adjacent Transformative
Do the same
thing better.
Nearby product,
market, or method.
Start something
entirely new.
Regional

optimizations.
Innovation, go-to-
market strategies.
Reinvent the
business model.
• Get there faster
• Smaller batches
• Solution, then testing
• Increased accountability
• Customer development
• Test similar cases
• Parallel deployment
• Analytics & cycle time
• Fail fast
• Skunkworks/R&D
• Focus on the search
• Ignore the current
model & margins
Another way to look at it
Core Adjacent Transformative
Know the problem
(customers tell you it)
Know the solution
(customers/regulations/
norms dictate it.)
Know the problem
(market analysis)
Don’t know the solution
(non-obvious innovation
confers competitive
advantage.)
Don’t know the problem
(just an emerging need/
change)
Don’t know the solution.
Waterfall:

Execution
matters
Agile/scrum:

Iteration
matters
Lean Startup:
Discovery
matters
Experiment with product, market,
and method.
Product
(new “what”)
Market
(new “who”)
Method
(new “how”)
3 kinds of
innovation
Engine as a service
http://www.nasa.gov/images/content/365835main_airplane_noise_qtd2_3024x2016.jpg
Engine as a service
“Efficiency is tied to
analytics. We’ll still look
for new materials, or for
the physics of devices,
but the analytics ... is
what’s really untapped.”
Current

state
Business
optimization
(five mores)

Product,

market,

method
innovation

Business
model
innovation

You can convince
executives of this
because some of it
is familiar.
This terrifies them
because it eats the
current business.
A three-maxima model
of enterprise innovation
Improvement Adjacency Remodeling
Do the same,

only better.
Explore what’s

nearby quickly
Try out new

business models
Lean approaches apply, but the metrics vary widely.
Sustain/

core
Innovate/

adjacent
Disrupt/

transformative
Sustaining Adjacent Disruptive
Next year’s car Electric car,

same dealer
On-demand, app-based

car service
Sustaining
innovation
is about
more of
the same.
(says Sergio Zyman)
More things
To more people
For more money
More often
More efficiently
Supply chain optimization
Per-transaction cost reduction
Loyal customer base that returns
Demand prediction, notification
Maximum shopping cart
Price skimming/tiering
Highly viral offering
Low incremental order costs
Inventory increase
Gifting, wish lists
Blizzard extends the
lifespan of WOW
Early

adopters
Rapid

growth
Market

saturation
The infamous S-curve
(Product lifecycle, Bass diffusion curve, etc.)
Blizzard extends the
lifespan of WOW
Blizzard extends the
lifespan of WOW
Fixing this: sustaining growth with novelty
Product & market innovation
(“New & improved!”)
Blizzard extends the
lifespan of WOW
WOW
Burning

Crusade
Wrath of

the Lich King
Mists of

PandariaCataclysm
Warlords of
Draenor
Most of your
innovation will
be adjacent or
sustaining.
Question marks!
(low market share,
high growth rate)
May be the next big thing.
Consumes investment, but
will require money to
increase market share.
Stars!
(high growth rate,
high market share)
What everyone wants. As
market invariably stops
growing, should become
cash cows.
Dogs!
(low market share,
low growth rate)
Barely breaks even, may
be a distraction from better
opportunities. Sell off or
shut down.
Cash cows!
(high market share,
low growth rate)
Boring sources of cash, to
be milked but not worth
additional investment.

Growthrate
Market share
Pivot to 

increase

market

share

through

virality,

attention
Pivot to

increase growth

rate through

disruption
Pivot to

redefine problem/

solution through

empathy
Milk with

revenue

optimization as

growth slows
If you don’t like
this, go launch
a startup.
Software, experimentation, and
iterative cycles of learning help you
get to the local maximum better and
faster. That’s a good thing.
But it’s not the only thing.
Adjacent innovation is about changing
one part of the model in a way that
alters the value network.
Amazon Web Services and the
server value network
Server computing
• Density
• Heat
• GHz
• MIPS
Cloud computing
• Instances
• Objects
• Spinup time
• Scaleout
Capex, financing,
TCO, ROI
Opex, demand, time
to result
CIO, enterprise IT CTO, coder, app owner,
line of business, startup
Value

criteria
Money
Buyer
Adjacent product
to the same
market in the
same way
Selling the same product to an
adjacent market in the same
way.
Of P&G’s 38 brands, only 19 were sold in Asia as of 2011
Market expansion is seldom selling the same thing to new people. In
Asia, P&G needed to
Align pricing with novelty (prestige, mass-tige, over-the-counter)
Change consumer expectations (moving from dilutes to
concentrates)
Adjust positioning and ingredients such as white fungus, ginseng,
and the parasitic cordyceps
Selling the same product to the same
market in a new way.
The biggest innovation in
logistics of the 20th century.
http://www.flickr.com/photos/photohome_uk/1494590209
Changing the method of
C2C classifieds
A blend of who, what, how
Classified C2C sales (same
“what”)
Strictly for Japanese women
(targeted “how”)
New how (phone is capture,
display, payment, transaction)
Did 100 interviews w/target
users before launch
Key insight: Japanese women
sell their entire wardrobe
twice in their lives
5,000 and 10,000 sales in first
month
10% commission fee
Average price of items is
pretty low, at around 2,000 to
3,000 yen (or $22 to $34)
Not an auction: seller decides
price
Mobile-only model
Phone is payment, storefront,
and even a way for sellers to
build their catalog
http://www.sffashtech.com/2012/10/10/a-free-market-fashion-app-exclusively-for-women-japan/
Selling the same product to the
same market in a new way.
(At this point, observant Intrapreneurs
should be asking, should P&G be in
the house cleaning business?
And that would be transformative.)
Transformative innovation is about
taking a leap, changing more than one
dimension simultaneously in search of
a new business model.
If sustaining, incremental innovation
produces linear growth, then
disruptive, transformative innovation
produces exponential growth.
Transformative isolation:
Skunkworks
Transformative
incubation:
Metlife Infinity
Significant market
850K full-time law enforcement officers in the
US; 700K state/local; 525K patrol officers
130M incident reports/y. 70M new incidents;
200K involve use of force
Only 31% of local police agencies keep
computer files on use-of-force incidents
Strong product benefits
Exonerates the officer 96% of the time.
47% percent increase in charges and
summons (2007)
Patrol officers spend 15-25% of their time
writing incident reports, recorded evidence
reduces this by 22%, meaning 50m more on
patrol
Challenges
New business model
Pricing unclear
SaaS offering
Compliance and governance
Unions, regulation, chain of evidence
Changing the current model (radio is
everything)
Transformative incubation:
Taser evidence.com
Part four:
What works for large companies
(A bagful of tricks from agitators in companies of all sizes.)
The job of an intrapreneur is to
identify an adjacent market, product,
or method that conforms to
organizational filters.
It is not to improve the current
product, market, or method.
Also: a pariah.
Successful innovators share certain attributes.
Bad listener: Wilfully ignore feedback from your best customers.
Cannibal: If successful, destroying existing revenue streams.
Job killer: Automation & lower margins are your favorite tools.
Security risk: Advocate of transparency, open data, communities.
Narcissist: Worry constantly about how you’ll get attention.
Slum lord: Sell to those with less money, deviants, and weirdos.
Know what
kind of
innovation
you’re
after.
New
Current
Current New
Market
Product
Penetrate:

Increase revenues,
market share, product
quality, brand
differentiation.
Marketing.
Market development:

Sell existing products
to new markets,
segments, uses.
Export & license.
Product
development:

Invent new products
for your market. R&D,
enhancements.
Acquisition.
Startup:

New products for new
markets. New rules,
business units,
organizational
structure. Innovation.
Based on H. Igor Ansoff’s matrix
Increased
risk
of political fallout (and
great success!)
Innovation portfolios at big companies
Core Adjacent Transformative
70% 20% 10%
Investment
70%20%10%
Return
Use outliers and missed searches to
hunt for good ideas & adjacencies
(Multi-billion-dollar hygiene product company)
1/8 men have an incontinence issue. 1/3
women do.
When search results show a significant
number of men searching, this suggests the
adjacent (male) market is underserved.
Frame it like a study
Product creation is almost
accidental.
Unlike a VC or startup, when
the initiative fails the
organization still learns.
http://www.flickr.com/photos/creative_tools/8544475139
When in doubt, collect data
From tackling the FTA rate to
visualizing the criminal justice
supply chain.
Use data to create a taste for
data
Sitting on Billions of rows of
transactional data
David Boyle ran 1M online surveys
Once the value was obvious to
management, got license to dig.
4” e-ink display with
name and specialty.
Badge scans barcode and
gets specs; checks
inventory; enters data on a
touch screen.
Smart Badge
Today: Workers see their own
productivity.

Coming soon: comparing
yourself to 400,000 other
employees.

Ultimately: Learning what
(and who) works well.
Data Exhaust
Tesco connects
its workforce
Don’t just collect data,
chase it.
Understand hidden
constraints
That pencil story is a myth.
Graphite is conductive and
explosive. The Minimum
Viable Product is Viable for a
reason.
Know what has to
be built in-house
SAP integration
Employment law
http://www.flickr.com/photos/bootbearwdc/1243690099/
Think subversively.
Everything’s an excuse to experiment
Find other ways
to collect data;
everything is an
experiment.
Run it as a consulting business first.
(Just don’t get addicted to it. Your goal is to
learn and overcome integration challenges and
find the 20% of features that 80% of the market
will pay for.)
Convince your boss she asked for this
Draw a new line
Pivot or

give up
Try again
Success!
Did we move
the needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:

find a
commonality
Without
data: make a
good guess
Find a
potential
improvement
Draw a line

in the sand
Pick a KPI
Focus on the desired behavior, not just
the information.
http://www.psychologytoday.com/blog/yes/
200808/changing-minds-and-changing-towels
26% increase in towel
re-use with an appeal
to social norms; 33%
increase when tied to
the specific room.
Energy Conservation “Nudges” and Environmentalist
Ideology: Evidence from a Randomized Residential Electricity
Field Experiment - Costa & Kahn 2011
The effectiveness of energy
conservation “nudges” depends on
an individual’s political ideology ...
Conservatives who learn that their
consumption is less than their
neighbors’ “boomerang” whereas
liberals reduce their consumption.
Take baby steps.
Netflix
Tesla
http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character
limit isn’t arbitrary. It’s
constrained by the size
of SMS (160
characters) and
username (20
characters.)
http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/
sms_screen_twitter_activity_stream_270x405.png
Figure out how to translate it back to a
simple model that fits the company’s
existing value model.
If your company dies, this is why.
Software
Platform
Merchandising
User-generated content
Marketplace
Media/content
Service
Oracle’s accounting suite
Amazon’s EC2 cloud
Thinkgeek’s retail store
Facebook’s status update
AirBnB’s list of house rentals
CNN’s news page
A hairstylist
Product

type
What the startup does in
return. May be a product
or service; may be
hardware or software;
may be a mixture.
One-time transaction
Recurring subscription
Consumption charges
Advertising clicks
Re-sale of user data
Donation
Single purchase from Fab
Monthly charge from Freshbooks
Compute cycles from Rackspace
PPC revenue on CNET.com
Twitter’s firehose license
Wikipedia’s annual campaign
Revenue

model
How the startup extracts
money from its visitors,
users, or customers.
Paid advertising
Search Engine Mgmt.
Social media outreach
Inherent virality
Artificial virality
Affiliate marketing
Public relations
App/ecosystem mkt.
Banner on Informationweek.com
High pagerank for ELC in kid’s toys
Active on Twitter i.e. Kissmetrics
Inviting team member to Asana
Rewarding Dropbox user for others’ signups
Sharing a % of sales with a referring blogger
Speaker submission to SXSW
Placement in the Android market
Acquisition

channel
How the visitor,
customer, or user finds
out about the startup.
Hosted service
Digital delivery
Physical delivery
Salesforce.com’s CRM
Valve purchase of desktop game
Knife shipped from Sur La Table
Delivery

model
How the product gets to
the customer.
Simple purchase
Discounts & incentives
Free trial
Freemium
Pay for privacy
Free-to-play
Buying a PC on Dell.com
Black Friday discount, loss leader, free ship
Time-limited trial such as fitbit Premium
Free tier, relying on upgrades, like Evernote

Free account content is public, like Slideshare
Monetize in-app purchases, like Airmech
Selling

tactic
What the startup does to
convince the visitor or
user to become a paying
customer.
Acquisition

channel
Selling

tactic
Revenue

model
Product

type
Delivery

model
Business
aspect
Flipbook

page(s)
Inherent virality.
Artificial virality.
Sharing files with others.
Free storage when others sign up.
Freemium.
Limited-capacity accounts are free;
subscribe when you need more.
Recurring subscription.
$99/year, monthly fees, enterprise
tiers.
Platform.
Storage-as-a-service with APIs,
collaboration, synchronization tools.
Hosted service.
Digital delivery.
Cloud storage, web interface.
Desktop client software.
Dropbox example
Use the right proxies for B2B products
Stage Startup metrics Intrapreneur metrics
Empathy
Customers interviewed (needs &
solutions), assumptions quantified, TAM,
monetization possibility
Non-customers interviewed; assumptions
quantified, constraints identified, TAM,
disruption potential
Stickiness Churn, engagement
Support tickets, integration time, call
center data, delays
Virality Viral coefficient, viral cycle time
Net Promoter Score, referrals, case study
willingness
Revenue Attention, engagement
Billable activity; signed LOIs; pilot
programs; after-development profitability
Scale Automation Contribution, training costs, licensing
Tomorrow’s company: Running parallel businesses
Innovation Sustaining/core Adjacent
Transformative/
disruptive
Core action
Optimizing/
improving
Experimenting
Searching/
inventing
Focus on Known metrics Risk removal
Assumption
validation
Which will live
Within current
business unit
Incubated, then
integrated
As new/separate
entities
Problem is Known Known Unknown
Solution is Known Unknown Unknown
How to build a message map
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
People who want to drive
Prospective car buyers
People looking for a hybrid
Honda Civic Hybrid owners
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
“Isn’t it time you got out of the
city?” campaign showing how cars
make nature accessible & ridiculing
urban hipsters.
Ads showing how cars are needed
any time (pregnancy, errands, urgent
business) and how a car is a
“personal assistant.”
Urgency (“every time you drive a
non-hybrid car you kill the planet a
little”) and testimonials from buyers
who’ve saved money.
Honda branding ads and model-
specific promotions.
Follow-up satisfaction campaign to
encourage buyers to tell their friends
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
Those who don’t need cars
• I’m too young to drive
• I’m too old to drive
• I can walk or take public
transit
Car users who won’t buy
• It’s too expensive for me
• I will use a shared car service
• It’ll get stolen
Those who won’t buy hybrids
• Hybrids are gutless
• Batteries are toxic & explosive
• In the end it costs more than
it saves
I will buy another brand
• I buy domestic
• I’ve always driven a VW
• Toyotas are reliable
• I want something prestigious
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
Those who don’t need cars
• I’m too young to drive
• I’m too old to drive
• I can walk or take public
transit
Car users who won’t buy
• It’s too expensive for me
• I will use a shared car service
• It’ll get stolen
Those who won’t buy hybrids
• Hybrids are gutless
• Batteries are toxic & explosive
• In the end it costs more than
it saves
I will buy another brand
• I buy domestic
• I’ve always driven a VW
• Toyotas are reliable
• I want something prestigious
Sponsor a driving school
“Give the gift of driving”
campaign for grandparents.
Financing, cashback
Sell to carshares;
underscore their limitations
PR on dangers of commuting,
pedestrian deaths
Theft warranty, tracking
services, high-end locks
Independent tests,
standard metrics (0-60 in X)
Lab research, studies
ROI calculator;
replacement programs
Prove Honda hires US workers
“Time to leave Germany” ads
Spontaneous accel. stories
Premium brand (Acura)
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
“Isn’t it time you got out of the
city?” campaign showing how cars
make nature accessible & ridiculing
urban hipsters.
Ads showing how cars are needed
any time (pregnancy, errands, urgent
business) and how a car is a
“personal assistant.”
Urgency (“every time you drive a
non-hybrid car you kill the planet a
little”) and testimonials from buyers
who’ve saved money.
Honda branding ads and model-
specific promotions.
Follow-up satisfaction campaign to
encourage buyers to tell their friends
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
Those who don’t need cars
• I’m too young to drive
• I’m too old to drive
• I can walk or take public
transit
Car users who won’t buy
• It’s too expensive for me
• I will use a shared car service
• It’ll get stolen
Those who won’t buy hybrids
• Hybrids are gutless
• Batteries are toxic & explosive
• In the end it costs more than
it saves
I will buy another brand
• I buy domestic
• I’ve always driven a VW
• Toyotas are reliable
• I want something prestigious
Sponsor a driving school
“Give the gift of driving”
campaign for grandparents.
Financing, cashback
Sell to carshares;
underscore their limitations
PR on dangers of commuting,
pedestrian deaths
Theft warranty, tracking
services, high-end locks
Independent tests,
standard metrics (0-60 in X)
Lab research, studies
ROI calculator;
replacement programs
Prove Honda hires US workers
“Time to leave Germany” ads
Spontaneous accel. stories
Premium brand (Acura)
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
Hits
A metric from the early, foolish days of the Web.
Count people instead.
Page views
Marginally better than hits. Unless you’re displaying
ad inventory, count people.
Visits
Is this one person visiting a hundred times, or are a
hundred people visiting once? Fail.
Unique visitors
This tells you nothing about what they did, why they
stuck around, or if they left.
Followers/friends/
likes
Count actions instead. Find out how many followers
will do your bidding.
Time on site, or
pages/visit
Poor version of engagement. Lots of time spent on
support pages is actually a bad sign.
Emails collected
How many recipients will act on what’s in them?
Number of
downloads
Outside app stores, downloads alone don’t lead to
lifetime value. Measure activations/active accounts.
Example: a restaurant
• Empathy: Before opening, the owner first learns about the diners in its area, their
desires, what foods aren’t available, and trends in eating.
• Stickiness: Then he develops a menu and tests it out with consumers, making
frequent adjustments until tables are full and patrons return regularly. He’s giving
things away, testing things, asking diners what they think. Costs are high because
of variance and uncertain inventory.
• Virality: He starts loyalty programs to bring frequent diners back, or to encourage
people to share with their friends. He engages on Yelp and Foursquare.
• Revenue: With virality kicked off, he works on margins—fewer free meals, tighter
controls on costs, more standardization.
• Scale: Finally, knowing he can run a profitable business, he pours some of the
revenues into marketing and promotion. He reaches out to food reviewers, travel
magazines, and radio stations. He launches a second restaurant, or a franchise
based on the initial one.
Example: a software company
• Empathy: The founder finds an unmet need, often because she has a background in a particular
industry or has worked with existing solutions that are being disrupted.
• Stickiness: She meets with an initial group of prospects, and signs contracts that look more like
consulting agreements, which she uses to build an initial product. She’s careful not to commit to
exclusivity, and tries to steer customers towards standardized solutions, charging heavily for
custom features. She supports the customers directly from the engineering team until the product
is stable and usable.
• Virality: Product in hand, she asks for references from satisfied customers, and uses them as
testimonials. She starts direct sales, and grows the customer base. She launches a user group,
and starts to automate support. She releases an API, encouraging third-party development and
scaling potential market size without direct development.
• Revenue: She focuses on growing the pipeline, sales margins, and revenues while controlling
costs. Tasks are automated, outsourced, or offshored. Feature enhancements are scored based on
anticipated payoff and development cost. Recurring license and support revenue becomes an
increasingly large component of overall revenues.
• Scale: She signs deals with large distributors, and works with global consulting firms to have them
deploy and integrate her tool. She attends trade shows to collect leads, carefully measuring cost of
acquisition against close rate and lead value.
Empathy
Get inside their head.
How to avoid leading the witness
Don’t tip your hand
Avoid biased wording, preconceptions,
or a giveaway appearance. Word your
surveys carefully to be neutral.
Make the question real
Get them to purchase. Ask them to pay. Demand real introductions. Or ask them “how
many of your friends would say X” to avoid self-effacement
Keep digging
Ask “why” several times. Leave lingering, uncomfortable pauses in the conversation and let
them fill them.
Look for other clues
Have a colleague make notes of when they react, or of their body language.
Mobile app model:

Localmind hacks Twitter
• Stage: Empathy
• Model: UGC/mobile
• Real-time question and answer platform tied to locations.
• Needed to find out if a core behavior—answering questions about a place—
happened enough to make the business real
Localmind hacks Twitter
• Before writing a line of code, Localmind was concerned that people would never
answer questions.
• This was their biggest risk: if questions went unanswered users would have a
terrible experience and stop using Localmind.
• Ran an experiment on Twitter
• Tracked geolocated tweets in Times Square
• Sent @ messages to people who had just tweeted, asking questions about the
area: how busy is it; is the subway running on time; is something open; etc.
• The response rate to their tweeted questions was very high.
• Good enough proxy to de-risk the solution, and convince the team and
investors that it was worth building Localmind.
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
Ask what existing
brands come to
mind in an industry
Market alongside
them? Address
competitors?
Choose partners?
Ask how customers
try to find a product
or service
Help you plan
marketing
campaigns and
choice of media
Ask what kind of
money people
spend on a problem
Shape your pricing
strategy
Test which tagline or
unique value
proposition
resonates best with
customers
Choose the
winning one, or just
take that as advice
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
• Design the survey Demographic
segmentation
questions
Quantifiable
answers to your
research problem
Qualitative, open-
ended feedback
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
• Design the survey
• Test it (you’ll always have mistakes)
• Know why you’re doing a survey in the first place
• Design the survey
• Test it (you’ll always have mistakes)
• Send it out
Audience plea
Creating an answers-at-scale campaign
Via your NW To a paid list As an ad campaign
Beware of
respondent bias,
misrepresentation of
the larger market
Beware of
spamminess, low
open rates
Name the problem
Give the solution or
unique value
“Are you a single
mom? Take this brief
survey and help us
address a big
challenge.”
“Can’t sleep? We’re
trying to fix that, and
want your input.”)
“Our accounting
software automatically
finds tax breaks. Help
us plan the product
roadmap.”
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
• Design the survey
• Test it (you’ll always have mistakes)
• Send it out
• Collect the results
• Analyze the data
Were you able to capture the attention of the
market? Did they click on your ads and links?
Which ones worked best?
Are you on the right track? What decisions can
you now make with the data you’ve collected?
Will people try out your solution/product? How
many of your respondents were willing to be
contacted? How many agreed to join a forum or
a beta? How many asked for access in their
open-ended responses?
By

segment!
When it’s time to move on
• Have you conducted enough quality customer interviews to feel confident that
I’ve found a problem worth solving?
• Do you understand your customer well enough?
• Do you believe your solution will meet the needs of customers?
Stickiness
The dogs like the dogfood.
Hits1995
Visits1997
Visitors1999
Conversions2002
Engagement2010
Who did you add? Where from? Why?
What did they do? How did it benefit?
Who did you lose? Why did they leave?
Days since last visit
1200
1000
800
600
400
200
0
1 2 3 4 5 6 7 8 9
Days since last engagement
January February
Disengaged
(>10 days)
25000
20000
15000
10000
5000
0
Numberofusers
When it’s time to move on
•	 Are people using the product as expected?
•	 Define an active user. What percentage of your users/customers is active?
Write this down. Could this be higher? What can you do to improve engagement?
•	 Evaluate your feature roadmap against the 7 questions to ask before building
more features. Does this change the priorities of feature development?
•	 Evaluate the complaints you’re getting from users. How does this impact
feature development going forward?
Virality
I told two friends.
------------------------------------------------------

Get your free private email at http://www.hotmail.com

------------------------------------------------------
Viral coefficient
Slides from New Media Manitoba Lean Analytics workshop, June 2015
v ≠ 1, pt  = δp0 (1 – vt+1) / (1 – v) + p0 
http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/
Viral coefficient
Or simpler
Users
Viral

coefficient
Churn &

abandonment
x - >
1
How to calculate it
• First calculate the invitation rate, which is the number of invites sent divided by the
number of users you have.
• Then calculate the acceptance rate, which is the number of signups or
enrollments divided by the number of invites.
• Then multiply the two together.
• Consider, for example
• Your 2,000 customers have sent out 5,000 invitations during their lifetime on
your site.
• Your invitation rate is 2.5.
• For every ten invitations received, one gets clicked.
• Your acceptance rate is 0.1.
• Multiply the two, and you have your viral coefficient: 0.25. Every customer you
add will add an addition 25% of a customer.
Virality stage:

Timehop’s content sharing
• Stage: Virality
• Model: Mobile app
• Social network around the past
• Focused on virality (but not necessarily the coefficient!)
The one metric that matters: content
sharing
• Focused on percent of daily active users that share their content
• Aiming for 20-30% of DAU sharing
“All that matters now is virality.
Everything else—be it press, publicity
stunts or something else—is like
pushing a rock up a mountain: it will
never scale. But being viral will.”
- Jonathan Wegener, co-founder
3 kinds of virality
•	 Inherent virality is built into the product, and happens as a function of use.
•	 Artificial virality is forced, and often built into a reward system.
•	 Word of mouth virality is simply conversations generated by satisfied users.
Laid, paid, made, afraid
Who is your
audience?
Drive-by visitors vs. serious evaluators; loyal users vs.
one-time buyers; lurkers vs. contributors. Have one
audience.
What do you want
them to do?
Tell five friends; sign up for a year’s subscription; write
a blog post on your behalf; invest some money. Just
be clear.
Why should they do
it?
Drive-by visitors versus serious evaluators; loyal
returning users versus one-time buyers; lurkers
versus contributors
Power or respect
among their peers;
appeal to reputation.
Money, or alternate
currency rewards; an
appeal to greed or
compensation.
Sex, attractiveness;
an appeal to desire.
Fear of missing out,
risk, loss, etc.; an
appeal to safety.
Made Paid Laid Afraid
Humans have four big motivators:
When it’s time to move on
• Are you using one of the three types of virality (inherent, artificial, word of mouth)
for your startup? Describe how. If virality is a weak aspect of your startup, write
down 3-5 ideas for how you could build more virality into your product.
• What’s your viral coefficient? Even if it’s below 1 (which it likely is), do you feel like
the virality that exists is good enough to help sustain growth and lower customer
acquisition costs?
• What’s your viral cycle time? How could you speed it up?
Revenue
Pour some of the money back into acquisition.
What’s a fitbit customer worth?
• The user can record their steps with a device in their pocket
• They can use it and sync data to the hosted application
• They can visit the portal to see their statistics
• They can manually enter sleep and food data
• They can buy the premium Fitbit offering
• Each of these is a different tier of engagement, and Fitbit could segment users
across these five segments when analyzing the effectiveness of a marketing
campaign or the volume of support e-mails.
When it’s time to move on
You’re making money
You’re sustainable
You’re tracking growth metrics
Revenue stage:

Backupify’s customer
lifecycle
• Stage: Revenue
• Model: SaaS
• Leading backup provider for cloud based data.
• The company was founded in 2008 by Robert May and Vik Chadha
• Has gone on to raise $19.5M in several rounds of financing.
You need a business model
Start with a systems diagram
What’s the core job to be done?
Start with the customer journey.
Consider the morning routine.
Wake
up
What happens here?
Start
workday
What job are you doing?
Personal
dimension
Social
dimension
Functional
aspects
Emotional
aspects
Related jobs to
be done
Functional
aspects
Emotional
aspects
Personal
dimension
Social
dimension
Main job to be
done
http://innovatorstoolkit.com/content/technique-1-jobs-be-done
Changes create new JTBD answers
Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953
+ = ?
Understand culture
What will it take for a change to really stick?
Whither

Estonia?
How about now?
Some stats
• 215M digital signatures
• 18 minutes to start a company, entirely online
• 1/3 of voting happens via the Internet
Get rid of the pens:

Digital signatures since 2000.
Only collect things once:

Tax forms 95% completed already.
Conclusions
“The most important figures that one
needs for management are unknown
or unknowable, but successful
management must nevertheless take
account of them.”
Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES
HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others
in the absence of data.
Now, a leader knows
what questions to ask.
Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko
Day two business models
Hardcore interviews
Maybe they don’t love you

like they said they do.
N
Your offering doesn’t make them want to

brag or their contact isn’t really a friend
N
Advocates can’t learn & convey

your message easily
N
They don’t trust you entirely
N
Woohoo! Scalable, viral,

explainable product!
Y
Get a meeting
Y
Grab the phone
Y
They pitch it
Y
Call them now?
Y
Intro to a friend?
Interview
B2B and the 80/20 rule
How to tell if you have a sustainable, repeatable B2B idea.
Target customer
Reachable, referenceable.
Find 10 prospects*
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
Target customer
Reachable, referenceable.
* 10 is an aribtrary
number. Just be sure
there are enough out
there to grow sustainably
from their actions.
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardize 80% of the offering
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Carefully learn what’s custom (& costly)Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Standardize, automate, patentCustom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Your margins,
differentiation, and scalable
growth come from this.
Without it, you’re either a consultant; or the
market isn’t ready; or the tech isn’t possible yet.
And that’s OK because you know.
Slides from New Media Manitoba Lean Analytics workshop, June 2015
The whole point of digital is personal
Segment 1
User segment

(who)
Segment 2
Segment 3
Goal

(what)
Goal 1
Goal 2
Goal 3
Motivation

(why)
Goal 1
Goal 2
Goal 3
http://www.thirdwunder.com/funnels-are-a-horrible-metaphor/Photo Credit: Patrick McGarvey
More like this.
Slides from New Media Manitoba Lean Analytics workshop, June 2015
The leaky bucket of subscriptions.
Photo by Jeffrey on Flickr - https://www.flickr.com/photos/jb912/7479464134
Bounces Non-

creators
Non-

payers
Churn
Signup

rate
Engagement

level
Conversion

rate
Visitors
Subscribers
Engaged users
Paying

customers
Leaky buckets are a
bad metaphor
Bounces Non-

creators
Non-

payers
Churn
Improve stickiness; call-to-action optimization;

A/B page testing; picking better traffic sources
Notifications; updates; reactivating users;

segmenting those who are engaged.
Usage caps, natural upselling;

premium features
Better support; credit card renewal;
naturally sustained features
Let them ask for an out
Detecting SaaS churn early without hurting cashflow
The tradeoff
Charge a monthly fee Charge annual fee up front
Find out if they hate it sooner, when
they cancel on first billing cycle.
No need to pay back CAC; cash you
can use right away.
Takes months to recoup the money
you spent acquiring them.
May be a zombie user who vanishes
when the year is up.
They’re happy, you keep your
money, goodwill for offering.
They’re unsatisfied, you made it
right, they tell you why, you learn.
The solution—because the goal is to
learn, not to trap customers.
Annual fee, with an out
1. Offer an annual, discounted fee.
2. After a couple of weeks, ask if
they’d like a refund.
The mobile app!
customer lifecycle!
Ratings
Reviews
Search
Leaderboards
Purchases
Downloads
Installs
Play
Disengagement
Reactivation
Uninstallation
Disengagement
Account"
creation
Virality
Downloads,"
Gross revenue
ARPU
App sales
Activation
Churn, CLV
In-app"
purchases
Appstore!
Incentivized
Legitimate
Fraudulent
Ratings!
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Targeting
matters.
Slides from New Media Manitoba Lean Analytics workshop, June 2015
It’s all about cabbages
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Start with the customer journey.
Consider the morning routine.
Wake
up
What happens here?
Start
workday
What job are you doing?
Personal
dimension
Social
dimension
Functional
aspects
Emotional
aspects
Related jobs to
be done
Functional
aspects
Emotional
aspects
Personal
dimension
Social
dimension
Main job to be
done
http://innovatorstoolkit.com/content/technique-1-jobs-be-done
Changes create new JTBD answers
Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953
+ = ?
I lied. Everyone is a tech company.
http://www.flickr.com/photos/puuikibeach/4789015423 http://www.flickr.com/photos/elcapitanbsc/3936927326
Cost of experiments: down. Cost of attention: way up.
Let’s pick on restaurants
for a while.
A line in the sand
Labor costs
Gross revenue
30%
24%
20%
Too costly?
Just right
Understaffed?
=
A leading indicator
http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970
50 reservations

at 5PM
250 covers

that night
(Varies by
restaurant.
McDonalds
≠ Fat Duck.)
http://www.flickr.com/photos/southbeachcars/6892880699
Restaurant MVP
Is tip amount a leading indicator of long-
term revenue?
Why does every table get the same
menu?
Is purple ink better?
http://tippingresearch.com/uploads/managing_tips.pdf
Slides from New Media Manitoba Lean Analytics workshop, June 2015

Weitere ähnliche Inhalte

Was ist angesagt?

Lean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean Analytics
 
Lean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Startup Co.
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics
 
OnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsLean Analytics
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Lean Analytics
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsLean Analytics
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Lean Analytics
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19Lean Analytics
 
Lean Analytics: A short summary
Lean Analytics: A short summaryLean Analytics: A short summary
Lean Analytics: A short summaryJan König
 
Big Data for the CMO
Big Data for the CMOBig Data for the CMO
Big Data for the CMOBruno Aziza
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)Lean Analytics
 
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Andreas Klinger
 
Measuring the ROI of content marketing
Measuring the ROI of content marketingMeasuring the ROI of content marketing
Measuring the ROI of content marketingJono Alderson
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Lean Analytics
 
Making Websites Work - Retail is Detail
Making Websites Work - Retail is Detail Making Websites Work - Retail is Detail
Making Websites Work - Retail is Detail Distilled
 
Lean startup metrics
Lean startup metricsLean startup metrics
Lean startup metricsStuart Eccles
 
Managing a Virtual Economy
Managing a Virtual EconomyManaging a Virtual Economy
Managing a Virtual EconomyWilliam Grosso
 
Be a Great Product Leader (Zynga 2016)
Be a Great Product Leader (Zynga 2016)Be a Great Product Leader (Zynga 2016)
Be a Great Product Leader (Zynga 2016)Adam Nash
 
The ultimate guide to data storytelling | Materclass
The ultimate guide to data storytelling | MaterclassThe ultimate guide to data storytelling | Materclass
The ultimate guide to data storytelling | MaterclassGramener
 

Was ist angesagt? (20)

Lean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basics
 
Lean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business Faster
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
 
OnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean Analytics
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
Lean Analytics: A short summary
Lean Analytics: A short summaryLean Analytics: A short summary
Lean Analytics: A short summary
 
Big Data for the CMO
Big Data for the CMOBig Data for the CMO
Big Data for the CMO
 
Metrics 101
Metrics 101Metrics 101
Metrics 101
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)
 
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
Startup Metrics, a love story. All slides of an 6h Lean Analytics workshop.
 
Measuring the ROI of content marketing
Measuring the ROI of content marketingMeasuring the ROI of content marketing
Measuring the ROI of content marketing
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)
 
Making Websites Work - Retail is Detail
Making Websites Work - Retail is Detail Making Websites Work - Retail is Detail
Making Websites Work - Retail is Detail
 
Lean startup metrics
Lean startup metricsLean startup metrics
Lean startup metrics
 
Managing a Virtual Economy
Managing a Virtual EconomyManaging a Virtual Economy
Managing a Virtual Economy
 
Be a Great Product Leader (Zynga 2016)
Be a Great Product Leader (Zynga 2016)Be a Great Product Leader (Zynga 2016)
Be a Great Product Leader (Zynga 2016)
 
The ultimate guide to data storytelling | Materclass
The ultimate guide to data storytelling | MaterclassThe ultimate guide to data storytelling | Materclass
The ultimate guide to data storytelling | Materclass
 

Ähnlich wie Slides from New Media Manitoba Lean Analytics workshop, June 2015

Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Lean Startup Co.
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
 
Social Media ROI: Necessary...Not Evil
Social Media ROI:  Necessary...Not EvilSocial Media ROI:  Necessary...Not Evil
Social Media ROI: Necessary...Not EvilArgyle Social
 
Basic Statistics Essay Examples
Basic Statistics Essay ExamplesBasic Statistics Essay Examples
Basic Statistics Essay ExamplesHelp Paper UK
 
How Technology And Globalization Have Enhanced Business...
How Technology And Globalization Have Enhanced Business...How Technology And Globalization Have Enhanced Business...
How Technology And Globalization Have Enhanced Business...Jennifer Moore
 
Discussion Questions Chapter 15Terms in Review1Define or exp.docx
Discussion Questions Chapter 15Terms in Review1Define or exp.docxDiscussion Questions Chapter 15Terms in Review1Define or exp.docx
Discussion Questions Chapter 15Terms in Review1Define or exp.docxedgar6wallace88877
 
Entering the Data Analytics industry
Entering the Data Analytics industryEntering the Data Analytics industry
Entering the Data Analytics industryGramener
 
Digital Marketing ROI at Blogworld NYC
Digital Marketing ROI at Blogworld NYCDigital Marketing ROI at Blogworld NYC
Digital Marketing ROI at Blogworld NYCChristopher Penn
 
7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing
7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing
7 Surefire Ways to Improve ROI for Your Email Program - BrightWave MarketingOnline Marketing Summit
 
Good College Essays For Applications
Good College Essays For ApplicationsGood College Essays For Applications
Good College Essays For ApplicationsTrina Locklear
 
Insights from Data: Overcoming Objections
Insights from Data: Overcoming ObjectionsInsights from Data: Overcoming Objections
Insights from Data: Overcoming ObjectionsGramener
 
The value of storytelling through data
The value of storytelling through dataThe value of storytelling through data
The value of storytelling through dataGramener
 
Crowdsourcing, Transparency and Results Based Charity Ratings
Crowdsourcing, Transparency and Results Based Charity RatingsCrowdsourcing, Transparency and Results Based Charity Ratings
Crowdsourcing, Transparency and Results Based Charity RatingsCharityNav
 
Email Marketing: Tools and Strategies - Blue Sky Factory
Email Marketing: Tools and Strategies - Blue Sky FactoryEmail Marketing: Tools and Strategies - Blue Sky Factory
Email Marketing: Tools and Strategies - Blue Sky FactoryOnline Marketing Summit
 
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18Mariia Bocheva
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Startup Co.
 
FAQ for the Predictive Testing of Opportunities
FAQ for the Predictive Testing of OpportunitiesFAQ for the Predictive Testing of Opportunities
FAQ for the Predictive Testing of OpportunitiesThe Inovo Group
 

Ähnlich wie Slides from New Media Manitoba Lean Analytics workshop, June 2015 (20)

Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
 
Social Media ROI: Necessary...Not Evil
Social Media ROI:  Necessary...Not EvilSocial Media ROI:  Necessary...Not Evil
Social Media ROI: Necessary...Not Evil
 
Basic Statistics Essay Examples
Basic Statistics Essay ExamplesBasic Statistics Essay Examples
Basic Statistics Essay Examples
 
How Technology And Globalization Have Enhanced Business...
How Technology And Globalization Have Enhanced Business...How Technology And Globalization Have Enhanced Business...
How Technology And Globalization Have Enhanced Business...
 
Intro.pptx
Intro.pptxIntro.pptx
Intro.pptx
 
Discussion Questions Chapter 15Terms in Review1Define or exp.docx
Discussion Questions Chapter 15Terms in Review1Define or exp.docxDiscussion Questions Chapter 15Terms in Review1Define or exp.docx
Discussion Questions Chapter 15Terms in Review1Define or exp.docx
 
Entering the Data Analytics industry
Entering the Data Analytics industryEntering the Data Analytics industry
Entering the Data Analytics industry
 
Digital Marketing ROI at Blogworld NYC
Digital Marketing ROI at Blogworld NYCDigital Marketing ROI at Blogworld NYC
Digital Marketing ROI at Blogworld NYC
 
Teaching students how to critically appraise organizational data
Teaching students how to critically appraise organizational dataTeaching students how to critically appraise organizational data
Teaching students how to critically appraise organizational data
 
7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing
7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing
7 Surefire Ways to Improve ROI for Your Email Program - BrightWave Marketing
 
Good College Essays For Applications
Good College Essays For ApplicationsGood College Essays For Applications
Good College Essays For Applications
 
Insights from Data: Overcoming Objections
Insights from Data: Overcoming ObjectionsInsights from Data: Overcoming Objections
Insights from Data: Overcoming Objections
 
The value of storytelling through data
The value of storytelling through dataThe value of storytelling through data
The value of storytelling through data
 
Crowdsourcing, Transparency and Results Based Charity Ratings
Crowdsourcing, Transparency and Results Based Charity RatingsCrowdsourcing, Transparency and Results Based Charity Ratings
Crowdsourcing, Transparency and Results Based Charity Ratings
 
Stats Essay Example
Stats Essay ExampleStats Essay Example
Stats Essay Example
 
Email Marketing: Tools and Strategies - Blue Sky Factory
Email Marketing: Tools and Strategies - Blue Sky FactoryEmail Marketing: Tools and Strategies - Blue Sky Factory
Email Marketing: Tools and Strategies - Blue Sky Factory
 
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair Croll
 
FAQ for the Predictive Testing of Opportunities
FAQ for the Predictive Testing of OpportunitiesFAQ for the Predictive Testing of Opportunities
FAQ for the Predictive Testing of Opportunities
 

Kürzlich hochgeladen

TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 

Kürzlich hochgeladen (17)

TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 

Slides from New Media Manitoba Lean Analytics workshop, June 2015

  • 1. Lean Analytics New Media Manitoba June, 2015 @acroll
  • 2. I can’t tell you how to succeed (If I could, I’d be starting your company instead of standing here.)
  • 3. I can tell you how to avoid failure (At least, some forms of it.)
  • 4. Percent of businesses that fail Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10 71%69%66%63%60%55%50%44%36%25% http://www.statisticbrain.com/startup-failure-by-industry/
  • 5. Still operating after 4 years Finance Insurance and Real Estate Education and Health Agriculture Services Wholesale Mining Manufacturing Construction Retail Transportation Communication and Utilities Information 37% 45% 47% 47% 49% 51% 54% 55% 56% 56% 58% http://www.statisticbrain.com/startup-failure-by-industry/
  • 6. Why firms die http://www.statisticbrain.com/startup-failure-by-industry/ Neglect, fraud, disaster 1% Lack of industry experience 13% Lack of managerial experience 34% Incompetence 52% Emotional Pricing Living too high for the business Nonpayment of taxes No knowledge of pricing Lack of planning No knowledge of financing No experience in record-keeping Poor credit granting practices Expansion too rapid Inadequate borrowing practices Carry inadequate inventory No knowledge of suppliers Wasted advertising budget
  • 8. Don’t sell what you can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
  • 9. The core of Lean is iteration.
  • 10. Waterfall approach You know the problem and the solution.
  • 12. Known set of requirements Known ways to satisfy them Spec Build Test Launch
  • 13. Agile methodologies Know the problem, find the solution
  • 15. Known set of requirements Unclear how to satisfy them Build Test LaunchViable?Problem
 statement Adjust Sprints Unknown set of requirements
  • 16. Lean approach First, know that you don’t know.
  • 17. Possible problem space Product/ market hypothesis Trial startup Product/ market hypothesis Trial startup Product/market hypothesis Trialstartup Product/market hypothesis Trialstartup You are herePIVOT
  • 19. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  • 20. Most startups don’t know what they’ll be when they grow up. Hotmail
 was a database company Flickr
 was going to be an MMO Twitter
 was a podcasting company Autodesk
 made desktop automation Paypal
 first built for Palmpilots Freshbooks
 was invoicing for a web design firm Wikipedia
 was to be written by experts only Mitel
 was a lawnmower company
  • 21. Gottfried von Leibniz was a geek just like you.
  • 22. First calculator (stepped reckoner) One of the first to recognize the importance of binary. “I thought again about my early plan of a new language or writing-system of reason, which could serve as a communication tool for all different nations..”
  • 23. The best of all possible worlds is the one in which the fewest starting conditions produce the greatest variety of outcomes.
  • 25. Analytics is the measurement of movement towards your business goals.
  • 26. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 27. Why now? First: High rate of change of digital technologies & channels.
  • 29. Times a song in “heavy rotation” is played daily 0 15 30 2007 2012 266
  • 30. Some fundamentals. Analytics, performance, aggregation, and the right metrics
  • 31. We’re bad at data.
  • 32. I have two kids. At least one of them is a girl.
  • 33. What are the chances the other is a boy?
  • 35. 2 of 3 (66%) are boys. GB GG BG
  • 37. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior
 changing What will you do differently based on the results you collect?
  • 38. The simplest rule bad
 metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
  • 39. Metrics help you know yourself. Acquisition Hybrid Loyalty 70%
 of retailers 20%
 of retailers 10%
 of retailers You are just like Customers that buy >1x in 90d Once 2-2.5
 per year >2.5
 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
  • 40. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  • 41. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  • 42. Rumsfeld on Analytics (Or rather, Avinash Kaushik channeling Rumsfeld) Things we know don’t
 know we know Are facts which may be wrong and should be checked against data. we don’t
 know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t
 know Are exploration which is where unfair advantage and interesting epiphanies live.
  • 43. MayAprMarFeb Slicing and dicing data Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate
 analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  • 44. Which of these two companies is doing better?
  • 45.   January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April   $8 $7 May       $9 How about this one?
  • 46. Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1   March $7 $6 $5     April $8 $7       May $9         Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 47. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 49. 1 10 100 1000 10000 Ice cream consumption Drownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
  • 50. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 51. A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 52. Is social action a leading indicator of donation? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
  • 55. A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 61. Why is Nigerian spam so badly written?
  • 62. Aunshul Rege of Rutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
  • 63. Turns out the word “Nigeria” is the best way to identify promising prospects.
  • 64. Nigerian spammers really understand their target market. They see past vanity metrics.
  • 66. “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” http://media.photobucket.com/image/einstein/derekabril/einstein_010.png
  • 67. “As simple as possible,
 but no simpler.” *(FYI, this is irony.)
  • 68. The Lean Analytics framework.
  • 69. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  • 70. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 71. Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
  • 72. Do you have a business model?
  • 76. Top-down Start with the whole planet, then narrow it down.
  • 78. Percent mobile phones by country
  • 80. Put this all together Metric/Assumption Value Population of Canada 34.88M Mobile phone percentage 80% Mobile phones in Canada 27,904M IOS percentage 15.5% IOS phones in Canada 4.32M IOS devices per account 3.1* IOS accounts we can sell to in Canada (this is our Total Addressable Market) 1.39M * I made this up. Don’t believe everything you read.
  • 81. How much is the TAM worth? Metric/Assumption Value TAM 1.39M Percent we will claim 5% Number of users 69,500 User lifetime 40 months Revenue per month $5 per month CLV $200 Expected total revenues $13.9M * I made this up. Don’t believe everything you read.
  • 82. Bottoms-up Start with one buyer, then add it up
  • 83. Bottoms-up Metric/Assumption Value Impressions (acquisition) 9,266,667 Percent that install 10% Installs (activation) 926,667 Percent still using after 30 days 25% Become regular users (retention) 231,667 Percent that buy 30% Buy premium version (revenue) 69,500 Percent that churn each month 2.5% Customer lifespan 40 Revenue per month 5 Customer lifetime value 200 TAM value $13,900,000
  • 84. Reality check Bottoms-up (TAM) Value Impressions (acquisition) 9.3 M impressions Top-down (TAM) Value IOS accounts we can sell to in Canada (this is our Total Addressable Market) 1.39 M users in Canada You Shall Not Pass.
  • 85. Six business model archetypes. E-commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market The business you’re in
  • 86. Can you move enough customers through this cycle to make more money than you spend? Make them refer others Retain them Get revenue from them Acquire customers Activate them http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
  • 87. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation
 rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
  • 88. Model + Stage = One Metric That Matters. One Metric
 That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  • 91. In a startup, focus is hard to achieve.
  • 92. Having only one metric addresses this problem.
  • 94. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  • 95. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 96. Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  • 98. Drawing some lines in the sand.
  • 99. A company loses a quarter of its customers every year. Is this good or bad?
  • 100. Not knowing what normal is makes you do stupid things.
  • 101. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  • 102. It’s oxygen You need customers to keep learning It’s a substitute for solvency PhotobyPaulMilleronFlickr.https://www.flickr.com/photos/94674772@N03/8788576498
  • 103. Baseline: 10% visitor engagement/day Fred Wilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  • 104. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 105. Baseline: Calculating customer lifetime 25%
 monthly churn 100/25=4
 The average customer lasts 4 months 5%
 monthly churn 100/5=20
 The average customer lasts 20 months 2%
 monthly churn 100/2=50
 The average customer lasts 50 months
  • 106. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
  • 107. Who is worth more? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
  • 109. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 110. The minimum needed to
 test the core idea Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient reads survey question Recipient responds to question Recipient installs mobile app Recipient creates account, profile Recipient sees survey results Recipient can edit profile, etc. 10-25%RESPONSERATE 70-90%RESPONSERATE
  • 111. Do AirBnB hosts get more business if their property is professionally photographed?
  • 112. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 113. 5,000 shoots per month by February 2012
  • 114. Hang on a second.
  • 115. Gut instinct (hypothesis) Professional photography helps AirBnB’s business SRSLY?
  • 116. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 117. “Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data:
 find a commonality Without data: make a good guess
  • 118. Circle of Moms: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
  • 119. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
  • 120. Part two:
 Why big companies
 need to change
  • 122. (http://csinvesting.org/2012/01/06/fortune-500-extinction/) F500 Life Expectancy Growth by entering a new business 95
 % fail Corporate Strategy Board 99
 % fail Clay Christensen 75 years 15 years 1950 2010...
  • 123. mikemace.com The slow death of a market leader. Revenue over time This is what most managers track. Note that sales keep rising (making you feel safe) until you run off the edge of the cliff. “Let’s cut prices to accelerate our growth.” “Time to enter the mainstream. Cut prices.” “We may miss the quarter. Let’s do a price promotion.” “That wasn’t supposed to happen. We’ll have to lay some people off. Gross margin percent Declining profit per unit (gross margin) is actually your best signal of trouble. The adoption curve Here’s where you actually are, but you don’t know it because you can’t draw the curve until after the market saturates. Early adopters Late adopters
  • 124. In other words, if your job is change you have your work cut out for you.
  • 125. Lesson one: Companies die because they fail to move to new business models.
  • 126. Clay Christensen, The Innovator’s Dilemma CostperMB $1000 $100 $10 $1 Time 14” M ainfram e 8” M inicom puter 5.25” Desktop 3.5” N otebook
  • 127. Technologies outstrip what the market needs, driven by feedback from the “best” current customer. Clay Christensen, The Innovator’s Dilemma $1000 $100 $10 $1 Time 8” 5.25” High end customer Low end customer
  • 128. The new market has different criteria for success, which are uninteresting to incumbents. Clay Christensen, The Innovator’s Dilemma $1000 $100 $10 $1 Time Storage
 capacity Portability
  • 129. Sometimes this has unintended consequences Clay Christensen, The Innovator’s Dilemma $1000 $100 $10 $1 Time Smaller disc size means less vibration impact, leading to greater density, increasing storage capacity
  • 130. Why now? First: High rate of change of digital technologies & channels.
  • 132. Times a song in “heavy rotation” is played daily 2007 2012 266
  • 133. This alone explains the collapse of modern print media. Circulation, annually Clicks, instantaneously Letters to the editor, weekly Hashtags, always
  • 134. Why now? Second: It’s no longer about whether you can build it—it’s about whether anyone will care.
  • 135. The Attention Economy “What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” (Computers, Communications and the Public Interest, pages 40-41, Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon
  • 136. Lit motors tests the risky part
  • 137. Lesson two: The difference between a rogue agent and a special operative is permission.
  • 138. If big firms can’t innovate, it’s this guy’s fault.
  • 139. Technology has radically changed the incremental cost of businesses.
  • 141. An economic order quantity of one. Crafted Mass- produced Automated Digital Quantity Few Many Some One Cost High Low Medium Free Lead time Small Large Medium None Self-service Medium None Some Lots Customization High None Some Lots • Cloud computing • Social media • 3D printing • Per-customer analysis • Mobile tracking • Etc... This is why software is eating the world.
  • 142. Sustainable competitive advantage allows for inertia and power to build up along the lines of an existing business model, which will soon die. Instead, seek transient competitive advantage. Rita Gunther McGrath, The End of Competitive Advantage
  • 143. Scale is now a liability. Compete on cycle time.
  • 144. CW&T make a pen
  • 147. Optimizing the probable means discounting the possible.
  • 149. This isn’t about a lack of resources.
  • 151. Blockbuster had a lot going for it.
  • 152. Plenty of inventory, of course. But that matters less than...
  • 153. ...market intelligence, customers, existing payment approval, and customer history.
  • 154. The problem was framing: Blockbuster thought it was in the video store management business. Netflix realized it was in the entertainment delivery business.
  • 159. • $1B invested in Nook • $475M operating loss in April 2013 • CEO gone First mover advantage happens long before the market emerges.
  • 161. Capital cycles don’t fit the short, iterative nature of startup uncertainty 12 month budgeting cycle; annual plan. Future based on past. Agile, scrum, lean iterations. Today’s model. No evidence about the future. Project Project Project Project Project Project Project Project Project Project Project Project (Requires budget insulation)
  • 163. The empty chair. Knowing this, how might you change the client/customer relationship?
  • 164. Can and must What triggers cultural change?
  • 165. “Using Cue, you can tell if someone has the ‘flu in 10 minutes.” http://tiltthewindmill.com/when-can-becomes-must/
  • 166. “Using Cue, you must tell if someone has the ‘flu in 10 minutes… …or Johnny can’t come on the school trip.”
  • 167. Part three: Three maxima of innovation
  • 168. When you’re a startup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
  • 169. Intrapreneur: Someone working to produce disruptive change in an organization that has already found a sustainable, repeatable business model.
  • 170. Business model vs. company stage Company size/age Early stage Big/incumbent B2B Target
 market B2C LessWoM
 Moreformaldecisions Slower cycle time
 More legacy constraints It is way too easy to mix these up. Intrapreneurs
  • 171. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 172. In a big company, analytics replaces opinion with fact.
  • 173. Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and profits than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011). 2011 MIT study of 179 large publicly traded firms
  • 174. The fundamental shift from Big Data Ask question Define
 schema Collect
 data Answer
 question Refine problem Collect data Ask question Emergent schema Explore data Answer
 question “Collect first; ask questions later.”
  • 175. Three kinds of innovation Sustain/core
 (optimizing for more of the same) Innovate/adjacent
 (introduce nearby product,
 market, or method) Disrupt/transformative
 (Fundamentally changing
 the business model) Improve along current metrics... ...or alter
 the rate of improvement Switch to a new value model Change the business
 model entirely
  • 176. Many models for enterprise innovation Core Adjacent Transformative Do the same thing better. Nearby product, market, or method. Start something entirely new. Regional
 optimizations. Innovation, go-to- market strategies. Reinvent the business model. • Get there faster • Smaller batches • Solution, then testing • Increased accountability • Customer development • Test similar cases • Parallel deployment • Analytics & cycle time • Fail fast • Skunkworks/R&D • Focus on the search • Ignore the current model & margins
  • 177. Another way to look at it Core Adjacent Transformative Know the problem (customers tell you it) Know the solution (customers/regulations/ norms dictate it.) Know the problem (market analysis) Don’t know the solution (non-obvious innovation confers competitive advantage.) Don’t know the problem (just an emerging need/ change) Don’t know the solution. Waterfall:
 Execution matters Agile/scrum:
 Iteration matters Lean Startup: Discovery matters
  • 178. Experiment with product, market, and method.
  • 180. Engine as a service
  • 182. “Efficiency is tied to analytics. We’ll still look for new materials, or for the physics of devices, but the analytics ... is what’s really untapped.”
  • 183. Current
 state Business optimization (five mores)
 Product,
 market,
 method innovation
 Business model innovation
 You can convince executives of this because some of it is familiar. This terrifies them because it eats the current business. A three-maxima model of enterprise innovation
  • 184. Improvement Adjacency Remodeling Do the same,
 only better. Explore what’s
 nearby quickly Try out new
 business models Lean approaches apply, but the metrics vary widely. Sustain/
 core Innovate/
 adjacent Disrupt/
 transformative
  • 185. Sustaining Adjacent Disruptive Next year’s car Electric car,
 same dealer On-demand, app-based
 car service
  • 186. Sustaining innovation is about more of the same. (says Sergio Zyman) More things To more people For more money More often More efficiently Supply chain optimization Per-transaction cost reduction Loyal customer base that returns Demand prediction, notification Maximum shopping cart Price skimming/tiering Highly viral offering Low incremental order costs Inventory increase Gifting, wish lists
  • 187. Blizzard extends the lifespan of WOW Early
 adopters Rapid
 growth Market
 saturation The infamous S-curve (Product lifecycle, Bass diffusion curve, etc.)
  • 189. Blizzard extends the lifespan of WOW Fixing this: sustaining growth with novelty Product & market innovation (“New & improved!”)
  • 190. Blizzard extends the lifespan of WOW WOW Burning
 Crusade Wrath of
 the Lich King Mists of
 PandariaCataclysm Warlords of Draenor
  • 191. Most of your innovation will be adjacent or sustaining. Question marks! (low market share, high growth rate) May be the next big thing. Consumes investment, but will require money to increase market share. Stars! (high growth rate, high market share) What everyone wants. As market invariably stops growing, should become cash cows. Dogs! (low market share, low growth rate) Barely breaks even, may be a distraction from better opportunities. Sell off or shut down. Cash cows! (high market share, low growth rate) Boring sources of cash, to be milked but not worth additional investment. Growthrate Market share Pivot to 
 increase
 market
 share
 through
 virality,
 attention Pivot to
 increase growth
 rate through
 disruption Pivot to
 redefine problem/
 solution through
 empathy Milk with
 revenue
 optimization as
 growth slows If you don’t like this, go launch a startup.
  • 192. Software, experimentation, and iterative cycles of learning help you get to the local maximum better and faster. That’s a good thing. But it’s not the only thing.
  • 193. Adjacent innovation is about changing one part of the model in a way that alters the value network.
  • 194. Amazon Web Services and the server value network Server computing • Density • Heat • GHz • MIPS Cloud computing • Instances • Objects • Spinup time • Scaleout Capex, financing, TCO, ROI Opex, demand, time to result CIO, enterprise IT CTO, coder, app owner, line of business, startup Value
 criteria Money Buyer
  • 195. Adjacent product to the same market in the same way
  • 196. Selling the same product to an adjacent market in the same way. Of P&G’s 38 brands, only 19 were sold in Asia as of 2011 Market expansion is seldom selling the same thing to new people. In Asia, P&G needed to Align pricing with novelty (prestige, mass-tige, over-the-counter) Change consumer expectations (moving from dilutes to concentrates) Adjust positioning and ingredients such as white fungus, ginseng, and the parasitic cordyceps
  • 197. Selling the same product to the same market in a new way. The biggest innovation in logistics of the 20th century. http://www.flickr.com/photos/photohome_uk/1494590209
  • 198. Changing the method of C2C classifieds A blend of who, what, how Classified C2C sales (same “what”) Strictly for Japanese women (targeted “how”) New how (phone is capture, display, payment, transaction) Did 100 interviews w/target users before launch Key insight: Japanese women sell their entire wardrobe twice in their lives 5,000 and 10,000 sales in first month 10% commission fee Average price of items is pretty low, at around 2,000 to 3,000 yen (or $22 to $34) Not an auction: seller decides price Mobile-only model Phone is payment, storefront, and even a way for sellers to build their catalog http://www.sffashtech.com/2012/10/10/a-free-market-fashion-app-exclusively-for-women-japan/
  • 199. Selling the same product to the same market in a new way.
  • 200. (At this point, observant Intrapreneurs should be asking, should P&G be in the house cleaning business? And that would be transformative.)
  • 201. Transformative innovation is about taking a leap, changing more than one dimension simultaneously in search of a new business model.
  • 202. If sustaining, incremental innovation produces linear growth, then disruptive, transformative innovation produces exponential growth.
  • 205. Significant market 850K full-time law enforcement officers in the US; 700K state/local; 525K patrol officers 130M incident reports/y. 70M new incidents; 200K involve use of force Only 31% of local police agencies keep computer files on use-of-force incidents Strong product benefits Exonerates the officer 96% of the time. 47% percent increase in charges and summons (2007) Patrol officers spend 15-25% of their time writing incident reports, recorded evidence reduces this by 22%, meaning 50m more on patrol Challenges New business model Pricing unclear SaaS offering Compliance and governance Unions, regulation, chain of evidence Changing the current model (radio is everything) Transformative incubation: Taser evidence.com
  • 206. Part four: What works for large companies (A bagful of tricks from agitators in companies of all sizes.)
  • 207. The job of an intrapreneur is to identify an adjacent market, product, or method that conforms to organizational filters. It is not to improve the current product, market, or method.
  • 208. Also: a pariah. Successful innovators share certain attributes. Bad listener: Wilfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.
  • 209. Know what kind of innovation you’re after. New Current Current New Market Product Penetrate:
 Increase revenues, market share, product quality, brand differentiation. Marketing. Market development:
 Sell existing products to new markets, segments, uses. Export & license. Product development:
 Invent new products for your market. R&D, enhancements. Acquisition. Startup:
 New products for new markets. New rules, business units, organizational structure. Innovation. Based on H. Igor Ansoff’s matrix Increased risk of political fallout (and great success!)
  • 210. Innovation portfolios at big companies Core Adjacent Transformative 70% 20% 10% Investment 70%20%10% Return
  • 211. Use outliers and missed searches to hunt for good ideas & adjacencies (Multi-billion-dollar hygiene product company) 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved.
  • 212. Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
  • 213. When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
  • 214. Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
  • 215. 4” e-ink display with name and specialty. Badge scans barcode and gets specs; checks inventory; enters data on a touch screen. Smart Badge Today: Workers see their own productivity. Coming soon: comparing yourself to 400,000 other employees. Ultimately: Learning what (and who) works well. Data Exhaust Tesco connects its workforce
  • 216. Don’t just collect data, chase it.
  • 217. Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
  • 218. Know what has to be built in-house SAP integration Employment law
  • 220. Everything’s an excuse to experiment
  • 221. Find other ways to collect data; everything is an experiment.
  • 222. Run it as a consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
  • 223. Convince your boss she asked for this Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a line
 in the sand Pick a KPI
  • 224. Focus on the desired behavior, not just the information. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011 The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption.
  • 228. Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size of SMS (160 characters) and username (20 characters.) http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
  • 229. Figure out how to translate it back to a simple model that fits the company’s existing value model. If your company dies, this is why.
  • 230. Software Platform Merchandising User-generated content Marketplace Media/content Service Oracle’s accounting suite Amazon’s EC2 cloud Thinkgeek’s retail store Facebook’s status update AirBnB’s list of house rentals CNN’s news page A hairstylist Product
 type What the startup does in return. May be a product or service; may be hardware or software; may be a mixture. One-time transaction Recurring subscription Consumption charges Advertising clicks Re-sale of user data Donation Single purchase from Fab Monthly charge from Freshbooks Compute cycles from Rackspace PPC revenue on CNET.com Twitter’s firehose license Wikipedia’s annual campaign Revenue
 model How the startup extracts money from its visitors, users, or customers. Paid advertising Search Engine Mgmt. Social media outreach Inherent virality Artificial virality Affiliate marketing Public relations App/ecosystem mkt. Banner on Informationweek.com High pagerank for ELC in kid’s toys Active on Twitter i.e. Kissmetrics Inviting team member to Asana Rewarding Dropbox user for others’ signups Sharing a % of sales with a referring blogger Speaker submission to SXSW Placement in the Android market Acquisition
 channel How the visitor, customer, or user finds out about the startup. Hosted service Digital delivery Physical delivery Salesforce.com’s CRM Valve purchase of desktop game Knife shipped from Sur La Table Delivery
 model How the product gets to the customer. Simple purchase Discounts & incentives Free trial Freemium Pay for privacy Free-to-play Buying a PC on Dell.com Black Friday discount, loss leader, free ship Time-limited trial such as fitbit Premium Free tier, relying on upgrades, like Evernote
 Free account content is public, like Slideshare Monetize in-app purchases, like Airmech Selling
 tactic What the startup does to convince the visitor or user to become a paying customer.
  • 231. Acquisition
 channel Selling
 tactic Revenue
 model Product
 type Delivery
 model Business aspect Flipbook
 page(s) Inherent virality. Artificial virality. Sharing files with others. Free storage when others sign up. Freemium. Limited-capacity accounts are free; subscribe when you need more. Recurring subscription. $99/year, monthly fees, enterprise tiers. Platform. Storage-as-a-service with APIs, collaboration, synchronization tools. Hosted service. Digital delivery. Cloud storage, web interface. Desktop client software. Dropbox example
  • 232. Use the right proxies for B2B products Stage Startup metrics Intrapreneur metrics Empathy Customers interviewed (needs & solutions), assumptions quantified, TAM, monetization possibility Non-customers interviewed; assumptions quantified, constraints identified, TAM, disruption potential Stickiness Churn, engagement Support tickets, integration time, call center data, delays Virality Viral coefficient, viral cycle time Net Promoter Score, referrals, case study willingness Revenue Attention, engagement Billable activity; signed LOIs; pilot programs; after-development profitability Scale Automation Contribution, training costs, licensing
  • 233. Tomorrow’s company: Running parallel businesses Innovation Sustaining/core Adjacent Transformative/ disruptive Core action Optimizing/ improving Experimenting Searching/ inventing Focus on Known metrics Risk removal Assumption validation Which will live Within current business unit Incubated, then integrated As new/separate entities Problem is Known Known Unknown Solution is Known Unknown Unknown
  • 234. How to build a message map
  • 235. I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  • 236. People who want to drive Prospective car buyers People looking for a hybrid Honda Civic Hybrid owners I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  • 237. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model- specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  • 238. People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  • 239. People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. Financing, cashback Sell to carshares; underscore their limitations PR on dangers of commuting, pedestrian deaths Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  • 240. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model- specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. Financing, cashback Sell to carshares; underscore their limitations PR on dangers of commuting, pedestrian deaths Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  • 241. Hits A metric from the early, foolish days of the Web. Count people instead. Page views Marginally better than hits. Unless you’re displaying ad inventory, count people. Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail. Unique visitors This tells you nothing about what they did, why they stuck around, or if they left. Followers/friends/ likes Count actions instead. Find out how many followers will do your bidding. Time on site, or pages/visit Poor version of engagement. Lots of time spent on support pages is actually a bad sign. Emails collected How many recipients will act on what’s in them? Number of downloads Outside app stores, downloads alone don’t lead to lifetime value. Measure activations/active accounts.
  • 242. Example: a restaurant • Empathy: Before opening, the owner first learns about the diners in its area, their desires, what foods aren’t available, and trends in eating. • Stickiness: Then he develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. He’s giving things away, testing things, asking diners what they think. Costs are high because of variance and uncertain inventory. • Virality: He starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. He engages on Yelp and Foursquare. • Revenue: With virality kicked off, he works on margins—fewer free meals, tighter controls on costs, more standardization. • Scale: Finally, knowing he can run a profitable business, he pours some of the revenues into marketing and promotion. He reaches out to food reviewers, travel magazines, and radio stations. He launches a second restaurant, or a franchise based on the initial one.
  • 243. Example: a software company • Empathy: The founder finds an unmet need, often because she has a background in a particular industry or has worked with existing solutions that are being disrupted. • Stickiness: She meets with an initial group of prospects, and signs contracts that look more like consulting agreements, which she uses to build an initial product. She’s careful not to commit to exclusivity, and tries to steer customers towards standardized solutions, charging heavily for custom features. She supports the customers directly from the engineering team until the product is stable and usable. • Virality: Product in hand, she asks for references from satisfied customers, and uses them as testimonials. She starts direct sales, and grows the customer base. She launches a user group, and starts to automate support. She releases an API, encouraging third-party development and scaling potential market size without direct development. • Revenue: She focuses on growing the pipeline, sales margins, and revenues while controlling costs. Tasks are automated, outsourced, or offshored. Feature enhancements are scored based on anticipated payoff and development cost. Recurring license and support revenue becomes an increasingly large component of overall revenues. • Scale: She signs deals with large distributors, and works with global consulting firms to have them deploy and integrate her tool. She attends trade shows to collect leads, carefully measuring cost of acquisition against close rate and lead value.
  • 245. How to avoid leading the witness Don’t tip your hand Avoid biased wording, preconceptions, or a giveaway appearance. Word your surveys carefully to be neutral. Make the question real Get them to purchase. Ask them to pay. Demand real introductions. Or ask them “how many of your friends would say X” to avoid self-effacement Keep digging Ask “why” several times. Leave lingering, uncomfortable pauses in the conversation and let them fill them. Look for other clues Have a colleague make notes of when they react, or of their body language.
  • 246. Mobile app model:
 Localmind hacks Twitter • Stage: Empathy • Model: UGC/mobile • Real-time question and answer platform tied to locations. • Needed to find out if a core behavior—answering questions about a place— happened enough to make the business real
  • 247. Localmind hacks Twitter • Before writing a line of code, Localmind was concerned that people would never answer questions. • This was their biggest risk: if questions went unanswered users would have a terrible experience and stop using Localmind. • Ran an experiment on Twitter • Tracked geolocated tweets in Times Square • Sent @ messages to people who had just tweeted, asking questions about the area: how busy is it; is the subway running on time; is something open; etc. • The response rate to their tweeted questions was very high. • Good enough proxy to de-risk the solution, and convince the team and investors that it was worth building Localmind.
  • 248. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place Ask what existing brands come to mind in an industry Market alongside them? Address competitors? Choose partners? Ask how customers try to find a product or service Help you plan marketing campaigns and choice of media Ask what kind of money people spend on a problem Shape your pricing strategy Test which tagline or unique value proposition resonates best with customers Choose the winning one, or just take that as advice
  • 249. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place • Design the survey Demographic segmentation questions Quantifiable answers to your research problem Qualitative, open- ended feedback
  • 250. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place • Design the survey • Test it (you’ll always have mistakes)
  • 251. • Know why you’re doing a survey in the first place • Design the survey • Test it (you’ll always have mistakes) • Send it out Audience plea Creating an answers-at-scale campaign Via your NW To a paid list As an ad campaign Beware of respondent bias, misrepresentation of the larger market Beware of spamminess, low open rates Name the problem Give the solution or unique value “Are you a single mom? Take this brief survey and help us address a big challenge.” “Can’t sleep? We’re trying to fix that, and want your input.”) “Our accounting software automatically finds tax breaks. Help us plan the product roadmap.”
  • 252. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place • Design the survey • Test it (you’ll always have mistakes) • Send it out • Collect the results • Analyze the data Were you able to capture the attention of the market? Did they click on your ads and links? Which ones worked best? Are you on the right track? What decisions can you now make with the data you’ve collected? Will people try out your solution/product? How many of your respondents were willing to be contacted? How many agreed to join a forum or a beta? How many asked for access in their open-ended responses? By
 segment!
  • 253. When it’s time to move on • Have you conducted enough quality customer interviews to feel confident that I’ve found a problem worth solving? • Do you understand your customer well enough? • Do you believe your solution will meet the needs of customers?
  • 254. Stickiness The dogs like the dogfood.
  • 255. Hits1995 Visits1997 Visitors1999 Conversions2002 Engagement2010 Who did you add? Where from? Why? What did they do? How did it benefit? Who did you lose? Why did they leave?
  • 256. Days since last visit 1200 1000 800 600 400 200 0 1 2 3 4 5 6 7 8 9 Days since last engagement January February Disengaged (>10 days) 25000 20000 15000 10000 5000 0 Numberofusers
  • 257. When it’s time to move on • Are people using the product as expected? • Define an active user. What percentage of your users/customers is active? Write this down. Could this be higher? What can you do to improve engagement? • Evaluate your feature roadmap against the 7 questions to ask before building more features. Does this change the priorities of feature development? • Evaluate the complaints you’re getting from users. How does this impact feature development going forward?
  • 258. Virality I told two friends.
  • 259. ------------------------------------------------------
 Get your free private email at http://www.hotmail.com
 ------------------------------------------------------ Viral coefficient
  • 261. v ≠ 1, pt  = δp0 (1 – vt+1) / (1 – v) + p0  http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ Viral coefficient
  • 263. How to calculate it • First calculate the invitation rate, which is the number of invites sent divided by the number of users you have. • Then calculate the acceptance rate, which is the number of signups or enrollments divided by the number of invites. • Then multiply the two together. • Consider, for example • Your 2,000 customers have sent out 5,000 invitations during their lifetime on your site. • Your invitation rate is 2.5. • For every ten invitations received, one gets clicked. • Your acceptance rate is 0.1. • Multiply the two, and you have your viral coefficient: 0.25. Every customer you add will add an addition 25% of a customer.
  • 264. Virality stage:
 Timehop’s content sharing • Stage: Virality • Model: Mobile app • Social network around the past • Focused on virality (but not necessarily the coefficient!)
  • 265. The one metric that matters: content sharing • Focused on percent of daily active users that share their content • Aiming for 20-30% of DAU sharing “All that matters now is virality. Everything else—be it press, publicity stunts or something else—is like pushing a rock up a mountain: it will never scale. But being viral will.” - Jonathan Wegener, co-founder
  • 266. 3 kinds of virality • Inherent virality is built into the product, and happens as a function of use. • Artificial virality is forced, and often built into a reward system. • Word of mouth virality is simply conversations generated by satisfied users.
  • 267. Laid, paid, made, afraid Who is your audience? Drive-by visitors vs. serious evaluators; loyal users vs. one-time buyers; lurkers vs. contributors. Have one audience. What do you want them to do? Tell five friends; sign up for a year’s subscription; write a blog post on your behalf; invest some money. Just be clear. Why should they do it? Drive-by visitors versus serious evaluators; loyal returning users versus one-time buyers; lurkers versus contributors Power or respect among their peers; appeal to reputation. Money, or alternate currency rewards; an appeal to greed or compensation. Sex, attractiveness; an appeal to desire. Fear of missing out, risk, loss, etc.; an appeal to safety. Made Paid Laid Afraid Humans have four big motivators:
  • 268. When it’s time to move on • Are you using one of the three types of virality (inherent, artificial, word of mouth) for your startup? Describe how. If virality is a weak aspect of your startup, write down 3-5 ideas for how you could build more virality into your product. • What’s your viral coefficient? Even if it’s below 1 (which it likely is), do you feel like the virality that exists is good enough to help sustain growth and lower customer acquisition costs? • What’s your viral cycle time? How could you speed it up?
  • 269. Revenue Pour some of the money back into acquisition.
  • 270. What’s a fitbit customer worth? • The user can record their steps with a device in their pocket • They can use it and sync data to the hosted application • They can visit the portal to see their statistics • They can manually enter sleep and food data • They can buy the premium Fitbit offering • Each of these is a different tier of engagement, and Fitbit could segment users across these five segments when analyzing the effectiveness of a marketing campaign or the volume of support e-mails.
  • 271. When it’s time to move on You’re making money You’re sustainable You’re tracking growth metrics
  • 272. Revenue stage:
 Backupify’s customer lifecycle • Stage: Revenue • Model: SaaS • Leading backup provider for cloud based data. • The company was founded in 2008 by Robert May and Vik Chadha • Has gone on to raise $19.5M in several rounds of financing.
  • 273. You need a business model Start with a systems diagram What’s the core job to be done?
  • 274. Start with the customer journey.
  • 275. Consider the morning routine. Wake up What happens here? Start workday
  • 276. What job are you doing? Personal dimension Social dimension Functional aspects Emotional aspects Related jobs to be done Functional aspects Emotional aspects Personal dimension Social dimension Main job to be done http://innovatorstoolkit.com/content/technique-1-jobs-be-done
  • 277. Changes create new JTBD answers Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953 + = ?
  • 278. Understand culture What will it take for a change to really stick?
  • 281. Some stats • 215M digital signatures • 18 minutes to start a company, entirely online • 1/3 of voting happens via the Internet
  • 282. Get rid of the pens:
 Digital signatures since 2000.
  • 283. Only collect things once:
 Tax forms 95% completed already.
  • 285. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 286. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 288. Once, a leader convinced others in the absence of data.
  • 289. Now, a leader knows what questions to ask.
  • 291. Day two business models
  • 293. Maybe they don’t love you
 like they said they do. N Your offering doesn’t make them want to
 brag or their contact isn’t really a friend N Advocates can’t learn & convey
 your message easily N They don’t trust you entirely N Woohoo! Scalable, viral,
 explainable product! Y Get a meeting Y Grab the phone Y They pitch it Y Call them now? Y Intro to a friend? Interview
  • 294. B2B and the 80/20 rule How to tell if you have a sustainable, repeatable B2B idea.
  • 295. Target customer Reachable, referenceable. Find 10 prospects* Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. * 10 is an aribtrary number. Just be sure there are enough out there to grow sustainably from their actions.
  • 298. Standardize, automate, patentCustom Custom Custom Custom Custom Custom Custom Custom Custom Custom Your margins, differentiation, and scalable growth come from this. Without it, you’re either a consultant; or the market isn’t ready; or the tech isn’t possible yet. And that’s OK because you know.
  • 300. The whole point of digital is personal Segment 1 User segment
 (who) Segment 2 Segment 3 Goal
 (what) Goal 1 Goal 2 Goal 3 Motivation
 (why) Goal 1 Goal 2 Goal 3
  • 303. The leaky bucket of subscriptions. Photo by Jeffrey on Flickr - https://www.flickr.com/photos/jb912/7479464134
  • 305. Bounces Non-
 creators Non-
 payers Churn Improve stickiness; call-to-action optimization;
 A/B page testing; picking better traffic sources Notifications; updates; reactivating users;
 segmenting those who are engaged. Usage caps, natural upselling;
 premium features Better support; credit card renewal; naturally sustained features
  • 306. Let them ask for an out Detecting SaaS churn early without hurting cashflow
  • 307. The tradeoff Charge a monthly fee Charge annual fee up front Find out if they hate it sooner, when they cancel on first billing cycle. No need to pay back CAC; cash you can use right away. Takes months to recoup the money you spent acquiring them. May be a zombie user who vanishes when the year is up.
  • 308. They’re happy, you keep your money, goodwill for offering. They’re unsatisfied, you made it right, they tell you why, you learn. The solution—because the goal is to learn, not to trap customers. Annual fee, with an out 1. Offer an annual, discounted fee. 2. After a couple of weeks, ask if they’d like a refund.
  • 309. The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU App sales Activation Churn, CLV In-app" purchases Appstore! Incentivized Legitimate Fraudulent Ratings!
  • 313. It’s all about cabbages
  • 315. Start with the customer journey.
  • 316. Consider the morning routine. Wake up What happens here? Start workday
  • 317. What job are you doing? Personal dimension Social dimension Functional aspects Emotional aspects Related jobs to be done Functional aspects Emotional aspects Personal dimension Social dimension Main job to be done http://innovatorstoolkit.com/content/technique-1-jobs-be-done
  • 318. Changes create new JTBD answers Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953 + = ?
  • 319. I lied. Everyone is a tech company.
  • 321. Let’s pick on restaurants for a while.
  • 322. A line in the sand Labor costs Gross revenue 30% 24% 20% Too costly? Just right Understaffed? =
  • 323. A leading indicator http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970 50 reservations
 at 5PM 250 covers
 that night (Varies by restaurant. McDonalds ≠ Fat Duck.)
  • 325. Is tip amount a leading indicator of long- term revenue?
  • 326. Why does every table get the same menu?
  • 327. Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf