This document provides an introduction to Lean Analytics for intrapreneurs. It begins with two key lessons: companies die when they fail to adopt new business models, and the difference between a rogue agent and special operative is permission. It then discusses Lean Analytics fundamentals like good metrics being understandable, comparative, ratios or rates, and behavior changing. It covers qualitative vs quantitative data, exploratory vs reporting analytics, and examples of leading metrics. The document emphasizes focusing on one metric that matters for a given business model and stage. It provides examples of analytics baselines for growth, engagement, churn, and calculating customer lifetime value.
7. Technologies
outstrip what
the market
needs, driven
by feedback
from the
“best”
current
customer.
$1
8”
5.25”
$10
end
High er
stom
cu
end
Low
mer
usto
c
$100
$1000
Clay Christensen, The Innovator’s Dilemma
Time
8. The new
market has
different criteria
for success,
which are
uninteresting to
incumbents.
$1
$10
$100
Storage
capacity
Portability
$1000
Clay Christensen, The Innovator’s Dilemma
Time
10. Three kinds of innovation
Improve along
current metrics...
Switch to a new
value model
Change the business
model entirely
...or alter
the rate of
improvement
Sustain/core
Innovate/adjacent
Disrupt/transformative
(optimizing for more of the same)
(introduce nearby product,
market, or method)
(Fundamentally changing
the business model)
17. Everyone’s idea is
the best right?
No data, no
learning.
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
24. A good metric is:
Understandable
Comparative
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparison is
context.
A ratio or rate
The only way to
measure
change and roll
up the tension
between two
metrics (MPH)
Behavior
changing
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
25. The
simplest
rule
If a metric won’t change how
you behave, it’s a
bad
metric.
h"p://www.flickr.com/photos/circasassy/7858155676/
26. Metrics help you know yourself.
Customers that
buy >1x in 90d
1-15%
15-30%
>30%
Then you are
in this mode
Your customers
will buy from you
You are
just like
70%
Acquisition
Once
Hybrid
2-2.5
20%
per year
of retailers
Loyalty
>2.5
10%
per year
of retailers
of retailers
Focus on
Low acquisition
cost, high checkout
Increasing return
rates, market share
Loyalty, selection,
inventory size
(Thanks to Kevin Hillstrom for this.)
28. Exploratory
Reporting
Speculative. Tries to find
unexpected or
interesting insights.
Source of unfair
advantages.
Predictable. Keeps you
abreast of the normal,
day-to-day operations.
Can be managed by
exception.
Cool.
Necessary.
29. Rumsfeld on Analytics
we know
Are facts which may be wrong and
should be checked against data.
we don’t
know
Are questions we can answer by
reporting, which we should baseline
& automate.
we know
Are intuition which we should
quantify and teach to improve
effectiveness, efficiency.
we don’t
know
Are exploration which is where
unfair advantage and interesting
epiphanies live.
know
Things we
don’t
know
(Or rather, Avinash Kaushik channeling Rumsfeld)
30. Slicing and dicing data
Active users
5,000
Cohort:
Comparison of
similar groups
along a timeline.
0
Jan
(this is the April cohort)
Feb
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
Mar
☀
☁
Apr
May
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
☀
☁
☀
☁
Multivariate
analysis
Changing several
things at once to
see which correlates
with a result.
37. Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.
Explaining the
past.
Leading
Forward-looking.
Number today that
predicts tomorrow;
reports the news.
Example: pipeline.
Predicting the
future.
38. Some examples
A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one file in one folder on one device
(ChenLi Wang)
Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
41. Correlated
Two variables that are
related (but may be
dependent on
something else.)
Ice cream &
drowning.
Causal
An independent variable
that directly impacts a
dependent one.
Summertime &
drowning.
42. A leading, causal metric
is a superpower.
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
47. Eric’s three engines of growth
Stickiness
Virality
Price
Approach
Keep people
coming back.
Make people
invite friends.
Spend money to
get customers.
Math that
matters
Get customers
faster than you
lose them.
How many they
tell, how fast they
tell them.
Customers are
worth more than
they cost.
48. Dave’s Pirate Metrics
AARRR
Acquisition
Activation
Retention
Revenue
Referral
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
49. The five stages
Stage
EMPATHY
STICKINESS
Gate
I’ve found a real, poorly-met need that a
reachable market faces.
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
50. Empathy stage:
Localmind hacks Twitter
Needed to find out if a core assumption—strangers answering
questions—was valid.
Ran Twitter experiment instead of writing code
Asked senders of geolocated Tweets from Times Square random
questions; counted response rate
Conclusion: high enough to proceed
51. Stickiness stage:
qidiq streamlines invites
Survey owner adds recipient to group
Survey owner adds recipient to group
Survey owner asks question
Recipient gets invite
Recipient installs mobile app
Recipient creates account, profile
Recipient can edit profile, etc.
Recipient reads survey question
Recipient responds to question
Recipient sees survey results
70-90% RESPONSE RATE
10-25% RESPONSE RATE
Survey owner asks question
Recipient reads survey question
Recipient responds to question
Recipient sees survey results
(Later, if needed…)
Recipient visits site; no password!
Recipient does password recovery
One-time link sent to email
Recipient creates password
Recipient can edit profile, etc.
54. Customer Acquisition Cost
paid
direct
search
wom
(Which means eye
charts like these.)
Viral coefficient
Viral rate
inherent
virality
VISITOR
Freemium/trial offer
Invite Others
Upselling
rate
Upselling
Enrollment
Capacity Limit
User
Disengaged User
Free user
disengagement
Paid
conversion
Engaged User
Tiering
Paying Customer
Support data
Reactivation
rate
Freemium
churn
Reactivate
Trial Over
Disengaged
Dissatisfied
Trial abandonment
rate
Resolution
Cancel
Cancel
Reactivate
Account Cancelled
Billing Info Exp.
Paid Churn Rate
FORMER USERS
User Lifetime Value
FORMER CUSTOMERS
Customer Lifetime Value
55. Model + Stage = One Metric That Matters.
The business you’re in
The stage you’re at
E-Com
SaaS
Mobile
2-Sided
Media
Empathy
Stickiness
Virality
Revenue
Scale
One Metric
That Matters.
UCG
61. Moz cuts down on metrics
SaaS-based SEO toolkit in the scale stage. Focused on net adds.
Net adds up:
Net adds flat:
Net adds down:
Was a marketing campaign successful?
Were customer complaints lowered?
Was a product upgrade valuable?
Can we acquire more valuable customers?
What product features can increase engagement?
Can we improve customer support?
Are the new customers not the right segment?
Did a marketing campaign fail?
Did a product upgrade fail somehow?
Is customer support falling apart?
62. Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
63. Ecommerce
Empathy
Stickiness
Virality
2-sided
market
Scale
Mobile
app
User-gen
content
Media
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
Inventory,
listings
CAC, shares,
SEM, sharing
reactivation
(Money from transactions)
Revenue
SaaS
Engagement, Downloads,
churn
churn, virality
Content,
spam
Traffic, visits,
returns
Inherent
virality, CAC
Invites,
sharing
Content
virality, SEM
WoM, app
ratings, CAC
(Money from active users)
Transaction,
CLV
Transactions,
commission
Upselling,
CAC, CLV
CLV,
ARPDAU
Affiliates,
white-label
Other
verticals
API, magic #, Spinoffs,
mktplace
publishers
(Money from ad clicks)
Ads,
donations
CPE, affiliate
%, eyeballs
Analytics,
user data
Syndication,
licenses
69. Baseline:
5-7% growth a week
• Are there enough people who really care
enough to sustain a 5% growth rate?
• Don’t strive for a 5% growth at the expense
of really understanding your customers
and building a meaningful solution
• Once you’re a pre-revenue startup at or
near product/market fit, you should have
5% growth of active users each week
• Once you’re generating revenues, they
should grow at 5% a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doing exceptionally well. If you
can only manage 1%, it's a sign
you haven't yet figured out
what you're doing.” At revenue
stage, measure growth in
revenue. Before that, measure
growth in active users.
Paul Graham, Y Combinator
70. Baseline:
10% visitor engagement/day
30%
of users/month use web or mobile app
10%
of users/day use web or mobile app
1%
of users/day use it concurrently
Fred Wilson’s social ratios
71. Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.
73. Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time to find
out whether your churn and revenue projections
are right
• Cashflow: You’re basically “loaning” the
customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a
CAC of only a third of your CLV will forces you
to verify costs sooner.
20 mo. CL
$30/month per
customer
$600 CLV
1/3 spend
$200 CAC
Now segment
those users!
75. Pick a KPI
Success!
Pivot or
give up
Draw a new line
Try again
Did we move the
needle?
Measure
the results
Draw a line
Find a potential
improvement
Without data:
make a good
guess
With data:
find a
commonality
Design a test
Hypothesis
Make changes
in production
76. Do AirBnB hosts
get more business
if their property is
professionally
photographed?
77. Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts
81. Pick a KPI
Success!
Pivot or
give up
Draw a new line
Try again
Did we move the
needle?
Measure
the results
Draw a line
Find a potential
improvement
Without data:
make a good
guess
With data:
find a
commonality
Design a test
Hypothesis
Make changes
in production
82. Without data: make a
good guess
With data:
find a commonality
“Gee, those
houses that do
well look really
nice.”
“Computer: What
do all the
highly rented
houses have in
common?”
Maybe it’s the
camera.
Camera model.
83. Landing page design
A/B testing
URL shortening
Publisher analytics
Cohort analysis
General analytics
Funnel analytics
SaaS analytics
User analytics
Spying on users
Influencer Marketing
Gaming analytics
User segmentation
User interaction
Customer satisfaction
KPI dashboards
84. Agenda
An introduction to
Lean Analytics (30m)
The challenges of
being big (15m)
When you have
support (30m)
The Lean Analytics
framework (30m)
A dose of
pragmatism (15m)
Metrics for innovation
portfolios (15m)
Break (15m)
Some non-tech
examples (10m)
Tools of the trade
(15m)
When it’s you against
the world (20m)
Break (15m)
86. It is way too easy to
mix these up.
Target
market
B2B
B2C
Early stage
Big/incumbent
Company size/age
Less WoM
More formal decisions
Intrapreneurs
Business model vs.
company stage
Slower cycle time
More legacy constraints
88. When product and market are known,
companies compete on how they do
things.
89. To get the incremental cost to zero,
companies competed on scale.
(Literally, an economy of scale)
90. Scale comes from process, IP, org
chart, capitalization.
All of these assume the future will be
like the past, only more so.
91. If a startup is an organization designed
to search for a sustainable, repeatable
business model, then an established
company is an organization designed
to perpetuate one.
93. Software is eating the world.
http://www.flickr.com/photos/ebolasmallpox/3733059220/
94. An economic order quantity
of one.
Crafted
Massproduced
Automated
Digital
Quantity
Few
Many
Some
One
Cost
High
Low
Medium
Free
Lead time
Small
Large
Medium
None
Self-service
Medium
None
Some
Lots
Customization
High
None
Some
Lots
This is why
software is
eating the
world.
• Cloud computing
• Social media
• 3D printing
• Per-customer
analysis
• Mobile tracking
• Etc...
95. Sustainable competitive advantage allows for
inertia and power to build up along the lines of
an existing business model, which will soon die.
Instead, seek transient competitive
advantage.
Rita Gunther McGrath, The End of Competitive Advantage
96. Scale is now a liability. Compete on
cycle time.
103. The problem was framing:
Blockbuster thought it was in the video
store management business. Netflix
realized it was in the entertainment
delivery business.
113. In other words, if your job is change you
have your work cut out for you.
114. 2011 MIT study of 179 large publicly traded firms
Companies that use data-driven
analytics instead of intuition have
5%-6% higher productivity and
profits than competitors.
Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven
Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).
116. Many models for enterprise innovation
Core
Adjacent
Transformative
Do the same
thing better.
Nearby product,
market, or method.
Start something
entirely new.
Regional
optimizations.
Innovation, go-tomarket strategies.
Reinvent the
business model.
• Customer development
• Test similar cases
• Parallel deployment
• Analytics & cycle time
• Fail fast
• Skunkworks/R&D
• Focus on the search
• Ignore the current
model & margins
• Get there faster
• Smaller batches
• Solution, then testing
• Increased accountability
117. Another way to look at it
Core
Adjacent
Transformative
Know the problem
(customers tell you it)
Know the solution
(customers/regulations/
norms dictate it.)
Know the problem
(market analysis)
Don’t know the solution
(non-obvious innovation
confers competitive
advantage.)
Don’t know the problem
(just an emerging need/
change)
Don’t know the solution.
Waterfall:
Execution
matters
Agile/scrum:
Iteration
matters
Lean Startup:
Discovery
matters
118. The Three Horizons
Core
Adjacent
Horizon 1 improves the
Horizon 2 extends the
current business operations business into new products,
in the next 12 months.
markets, or methods in the
next 3 years.
Those core businesses most
readily identified with the company
name and those that provide the
greatest profits and cash flow.
Maximize remaining value.
Emerging opportunities, including
rising entrepreneurial ventures
likely to generate substantial
profits in the future but that could
require considerable investment.
Transformative
Horizon 3 changes the
industry you’re in and your
value network in the next 6
years.
Ideas for profitable growth down the
road—for instance, small ventures
such as research projects, pilot
programs, or minority stakes in new
businesses.
http://www.mckinsey.com/insights/strategy/enduring_ideas_the_three_horizons_of_growth
122. A three-maxima model
of enterprise innovation
Business
optimization
(five mores)
Current
state
Business
model
innovation
Product,
market,
method
innovation
You can convince
executives of this
because some of it
is familiar.
This terrifies them
because it eats the
current business.
123. Improvement
Adjacency
Remodeling
Do the same,
only better.
Explore what’s
nearby quickly
Try out new
business models
Lean approaches apply, but the metrics vary widely.
Sustain/
core
Innovate/
adjacent
Disrupt/
transformative
124. More things
To more people
Sustaining
innovation
is about
more of
the same.
(says Sergio Zyman)
Inventory increase
Gifting, wish lists
Highly viral offering
Low incremental order costs
For more money
Maximum shopping cart
Price skimming/tiering
More often
Loyal customer base that returns
Demand prediction, notification
More efficiently
Supply chain optimization
Per-transaction cost reduction
125. Software, experimentation, and
iterative cycles of learning help you
get to the local maximum better and
faster. That’s a good thing.
But it’s not the only thing.
126. Adjacent innovation is about changing
one part of the model in a way that
alters the value network.
128. Selling the same product to an
adjacent market in the same
way.
Of P&G’s 38 brands, only 19 were sold in Asia as of 2011
Market expansion is seldom selling the same thing to new people. In
Asia, P&G needed to
Align pricing with novelty (prestige, mass-tige, over-the-counter)
Change consumer expectations (moving from dilutes to
concentrates)
Adjust positioning and ingredients such as white fungus, ginseng,
and the parasitic cordyceps
129. Selling the same product to the same in
The biggest innovation
logistics of the 20th century.
market in a new way.
http://www.flickr.com/photos/photohome_uk/1494590209
137. Cost of experiments: down.
http://www.flickr.com/photos/puuikibeach/4789015423
Cost of attention: way up.
http://www.flickr.com/photos/elcapitanbsc/3936927326
139. Empathy: find the need
Before opening, the owner first learns about the diners in her
area, their desires, what foods aren’t available, and trends in
eating.
Key metrics: Popular items;
frequent questions; before/after
dining patterns.
Reference: Emerging need.
140. Stickiness: confirm the need is met.
She develops a menu and tests it out with consumers,
making frequent adjustments until tables are full and patrons
return regularly. She’s giving things away, asking diners what
they think. Variance and uncertain inventory make costs high.
Key metrics: Customer loyalty;
recommendations; referrals;
endorsements; inventory turnover.
Reference: Business idea.
141. Virality: will it spread?
She starts loyalty programs to bring frequent diners back, or
to encourage people to share with their friends. She engages
on Yelp and Foursquare.
Key metrics: Customer loyalty;
recommendations; referrals;
endorsements.
Reference: Business positioning
142. Revenue: prove the business viability
With virality kicked off, she works on margins—fewer free
meals, tighter controls on costs, more standardization. She
focuses on the price of acquiring new customers.
Key metrics: Acquisition cost,
revenue per cover, capacity,
turnover.
Reference: Business model.
143. Scale: prove it’s a market
Knowing she can run a profitable business, she funnels
revenues into marketing and promotion. She reaches out to
food reviewers, travel magazines, and radio stations. She
launches a second restaurant, or a franchise.
Key metrics: Franchise health;
repeatability; problems escalated;
variance; franchise revenues.
Reference: Business plan.
144. A line in the sand
30%
Labor costs
Gross revenue
= 24%
20%
Too costly?
Just right
Understaffed?
145. A leading indicator
(Varies by
restaurant.
McDonalds
≠ Fat Duck.)
50 reservations
at 5PM
http://www.flickr.com/photos/mysticcountry/3567440970
250 covers
that night
http://www.flickr.com/photos/avlxyz/4889656453
152. When it’s you vs. the world.
(A bagful of tricks from agitators in companies of all sizes.)
153. The skills you need make you a pariah.
Successful innovators share certain attributes.
Bad listener: Willfully ignore feedback from your best customers.
Cannibal: If successful, destroying existing revenue streams.
Job killer: Automation & lower margins are your favorite tools.
Security risk: Advocate of transparency, open data, communities.
Narcissist: Worry constantly about how you’ll get attention.
Slum lord: Sell to those with less money, deviants, and weirdos.
154. New
Know what
kind of
innovation
you’re
Market
after.
Current
Based on H. Igor Ansoff’s matrix
Startup:
New products for new
!)
markets. News rules,
es
cc units,
business
su
at
organizational
re
g
nd
structure. Innovation.
(a
Market development:
Sell existing products
to new markets,
segments, uses.
Export & license.
lf
a
ll
a
Penetrate: itic
ol
Increase revenues,
fp
o
market share, product
sk
ri
ed
quality, brand
as
re
differentiation.
nc
I
Marketing.
Current
ut
o
Product
development:
Invent new products
for your market. R&D,
enhancements.
Acquisition.
Product
New
155. Frame it like a study
Product creation is almost
accidental.
Unlike a VC or startup, when
the initiative fails the
organization still learns.
http://www.flickr.com/photos/creative_tools/8544475139
156. When in doubt, collect data
From tackling the FTA rate to
visualizing the criminal justice
supply chain.
157. Use data to create a taste for
data
Sitting on Billions of rows of
transactional data
David Boyle ran 1M online surveys
Once the value was obvious to
management, got license to dig.
164. Run it as a consulting business first.
(Just don’t get addicted to it. Your goal is to
learn and overcome integration challenges and
find the 20% of features that 80% of the market
will pay for.)
165. Convince your boss she asked for this
Success!
Pick a KPI
Pivot or
give up
Draw a new line
Try again
Did we move
the needle?
Measure
the results
Draw a line
in the sand
Find a
potential
improvement
Without
data: make a
good guess
With data:
find a
commonality
Design a test
Make changes
in production
Hypothesis
166. Slaughter a sacred cow:
Prove a long-held assumption is
wrong and you’ve got people’s
attention.
Know what you’ll do with it ahead of
time.
170. Twitter’s 140-character
limit isn’t arbitrary. It’s
constrained by the size
of SMS (160 characters)
and username (20
characters.)
http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/
sms_screen_twitter_activity_stream_270x405.png
171. Figure out how to translate it back to a
simple model that fits the company’s
existing value model.
If your company dies, this is why.
172. Intrapreneurs often have to use proxies
Stage
Startup metrics
Intrapreneur metrics
Empathy
Customers interviewed (needs &
solutions), assumptions quantified,
TAM, monetization possibility
Non-customers interviewed;
assumptions quantified, constraints
identified, TAM, disruption potential
Churn, engagement
Support tickets, integration time, call
center data, delays
Viral coefficient, viral cycle time
Net Promoter Score, referrals, case
study willingness
Revenue
Attention, engagement
Billable activity; signed LOIs; pilot
programs; after-development
profitability
Scale
Automation
Contribution, training costs, licensing
Stickiness
Virality
173. When you have support.
(What companies like P&G, Cognizant, GE, and Motorola do with a
formal innovation program.)
174. Do you really have permission?
What resources do you have?
Staff, budget, unfettered access to customers?
What scope of change can you make?
Pricing, product, channel, branding?
176. Innovation portfolios at big companies
Return
Investment
Core
Adjacent
70%
20%
10%
20%
Transformative
10%
70%
177. 1.
Organizations’ structures emerge as a way
to optimize the current business model.
2.
Most innovations will come not from
product or market, but from method—
business model innovation.
3.
Innovation groups must exercise
organizational amnesia at the outset.
185. Find non-obvious adjacencies
POWER GRID
WHICH
FEEDS A
NEEDS
LIGHT AN
BULB
ELECTRICAL
GENERATOR
HAS A
TURBINE
LIKE A
TURNED
AROUND
BECOMES A
PLANE ENGINE
TRAIN ENGINE
REQUIRES
SPINS &
VIBRATES
LIKE AN
AND
LOOKS
LIKE A
SOFTWARE TO
CUT DOWN
TREES BETTER
MRI MACHINE
WIND TURBINE
186. Build an ecosystem
Canada’s largest directory
publishing and local
marketing services company
1.5M listings from 420K
SMB & national customers
Revenues >$1.2B
2,500 employees
Created third-party listing API
Took 8-10 mo (2009-10)
to get approval
API payoff happened 2y later
KPI evolution
Yahoo replaced Canadian
digital properties search
with the YellowAPI
Soft: Signups,
SDK, downloads
Improved SEO, Comscore
App usage,
deals signed
Functional prototype in
hours, testing in days, and
launching in weeks.
Faster time to partnerships
Budgets tripled in 2013
API calls
generated
API-generated
revenue
187. Five common models for
transformative innovation
Acquisition
Collaboration
Isolation
Buy promising startups
Crowdsource, work with
universities, suppliers, etc.
Create a separate group
with different conditions
Incubation
Internal startup ecosystem;
LoB are “investors”
Integration
The LoB does innovation
internally
188. Step five: Test by doing (experimentation
beats projection.)
189. Focus on
the model,
not the
plan
Demand
Amt
Growth Wk 1 Wk 2 Wk 3
People per day on sidewalk
200
4%
204
Percent that buy a glass
10%
5%
15% 20% 25%
Daily customers
20
31
Revenue
Price per cup
Profit per cup
Daily profit
43
225
56
$156 $216 $281
$5
$5
Cost of Goods
Cost per cup
216
$5
$5
30.6 41.5 52.8
$1
-2%
.98
.96
.94
$4
4.02 4.04 4.06
$80
125
175
228
190. A business plan is just what happens
when you drag the business model to
the right.
191. Designing an experiment
Problem, solution, and result hypothesis
Test strategy (PoC, survey, interviews, kickstarter, prototype, A/B, etc.)
Cohort & segment to be tested
Metric or assumption being tested
Timebox or total for test
Action you’ll take if you pass or fail
193. Qualcomm’s innovation model:
What was missing
Hypothesis
Unclear what
happened to
founders
Needed a
middle PoC
decision
Sustainability,
not feasibility
http://blogs.berkeley.edu/2013/01/28/
designing-a-corporateentrepreneurship-program-aqualcomm-case-study-part-1-of-2/
Experiment
Boot
camp
Idea generation
and selection
Existing models
Implement
Idea
advancement
POC
Biz
sustainability
Tech
feasibility
POC
Ideas
New
models
Open
innovation
Strategic
value
decision
Boot
camp
decision
End user/partner
desirability
Option
value
Implement
decision
Actions
Exit
value
Company crowd storm Small team designs & Company extracts value
conducts experiments
195. The Lean Analytics lifecycle of an Intrapreneur
Get buy-in
Political fallout
Find problems; don’t test demand.
Skip the business case, do analytics
Entitled, aggrieved
customers
Stickiness
Know your real minimum based on
expectations, regulations
Hidden “must haves”,
feature creep
Virality
Build inherent virality in from the
start; attention is the new currency
Luddites who don’t
understand sharing
Revenue
Consider the ecosystem, channels,
and established agreements
Channel conflict,
resistance, contracts
Hand the baton to others gracefully
Hating what happens
to your baby
Beforehand
Empathy
Scale
197. Core metrics
Business plan.
Metrics that matter
Assume it will improve.
• Return on investment
• Total cost of ownership
• Improvement in KPI
• Total served market
Product, market, and method will
remain the same
Examples: Redesigning
packaging; pricing
adjustment
198. Adjacent metrics
Business model.
Metrics that matter
Assume it will fail.
• Virality & word of mouth
• Early adopter stickiness
• Regulation
• Total addressable market
Your ultimate use case won’t be
what you think it is today.
Example: Mr. Clean
Magic Eraser
199. Transformative metrics
Business idea.
Assume it will fail.
You hope it will have the
consequences you want but
aren’t sure how.
Example: Headcam
recordings of all officers
Metrics that matter
• People I’ve talked to
• Prototype creation speed
• Assumptions validated
• Problems uncovered
• Technical feasibility
• Hidden constraints
200. Key points to clarify in an
innovation program
Hypothesis
Experimentation
• Articulate problems
• Frame known advantages
• Define the right filters
• Many idea sources
• Confirm funding (money,
• Prioritize riskiest
•
•
•
people, customer access)
Agree on analytical
framework
Balance market, product,
& method adjacencies
•
•
assumptions
Time-box assessment
stages
Test technology, demand,
and business feasibility
MVP, prototype, pilot, or
science as appropriate for
type of innovation
Implementation
• Temporary incubator
• Find a home or building
•
•
•
one
Keep innovators involved
Merge metrics with
existing business KPIs
Synchronize innovation
cycles with enterprise
cycles (budget, etc.)
Portfolio metrics; Gates and KPIs for each stage; mix of core, adjacent, and
disruptive innovation.
201. Hypothesis
Goals, constraints, context
Sourcing
Top-down
Bottom-up
Sourcing
Filtering
Biases
Strengths
Focus
Mandate
Outside-in
ost e
l alm as th
wil s w
this . thi
at
re a o
.
e th ange. here a d t
ot ch
N y
dt
nee
n
inl eck a ions I
erta P d
c
rat it.
MV of alte e to
ak
nch
m
bu
Boot camp
POC
Core
Optimization
Crosspollinate
to current
managers
Adjacent
De-risking
Best solution
Test/
validate
w/current
customers
Disruptive
Reframing
R&D
M&A
Grow
as a
distinct
business
Evaluation of the innovation program itself
Implementation
Socializing
Integration
Adoption by
existing line of
business.
Independence
Creation of a new
line of business.
203. Traction graphs
Your business model
The stage you’re at
... change often if
you’re doing it right.
Your one metric
So how do you track
that over time?
206. Use vanity to get to
meaningful metrics
Your goal is to produce
outcomes
If the outcomes require
action, and vanity motivates
actors, use it
But show how the vanity
metric is a leading indicator of
the real one
Web traffic
Activation
Cart
Size
Conversion
x
rate
Revenue
207. The three threes
Three
assumptions
What big bets are you making?
• “People will answer questions”
• “Organizers are frustrated with how to run conferences”
• “We'll make money from parents”
• “Amazon is reliable enough for our users.”
Three actions
to take
What are you doing to make these assumptions happen (or
identify they’re wrong and change course?)
• Product enhancements
• Marketing strategies
Three experiments
to run
• Feature tests
• Continuous deployment
• A/B testing
• Customer survey
209. The three threes
Many people will
answer questions
Three
assumptions
Three actions
to take
Three experiments
to run
Get more
people
Change
the UI
Increase
answer %
Test
timings
Test better
questions
Questions
from peers
210. The problem-solution canvas
The Goal is to Learn
CURRENT STATUS
• List key metrics you’re
LAST WEEK’S LESSONS LEARNED
AND ACCOMPLISHMENTS)
• What did you learn last week?
tracking, where they’re at, and
• What was accomplished?
compare with last few weeks
• On track: YES / NO?
How are things trending?
•
211. The problem-solution canvas
Problem #1 (put name here)
HYPOTHESIZED SOLUTIONS
• List possible solutions that you’ll start
working on next week. Rank them.
• Why do you believe each solution will
help you solve or complete solve the
problem?
METRICS / PROOF + GOALS
• Metrics you’ll use to measure whether
or not the solutions are doing what you
hoped (solving the problem)
• List proof (qualitative) you’ll use as well
• Define goals for the metric
Problem #2 (put name here)
HYPOTHESIZED SOLUTIONS
• List possible solutions that you’ll start
working on next week. Rank them.
METRICS / PROOF + GOALS
• Metrics you’ll use to measure whether
or not the solutions are doing what you
hoped (solving the problem)
216. Key points
Intrapreneurship is about adjacent or transformative innovation
Sustaining innovation focuses on the Five Mores, within the
current product, market, method, and business model.
Adjacent innovation may come from a new product, market, or
method, but the same business model
Disruptive innovation has different customers, KPIs, and models
The difference between a rogue agent and a special operative is
permission
Portfolios need framing, sourcing, filters, metrics, and socializing
Balancing isolation and integration, R&D and M&A is contentious
217. “The most important figures that one
needs for management are unknown
or unknowable, but successful
management must nevertheless take
account of them.”
Lloyd S. Nelson
218. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844