Weitere ähnliche Inhalte Ähnlich wie Harnessing the Power of Product Analytics by Dan Olsen (20) Kürzlich hochgeladen (20) Harnessing the Power of Product Analytics by Dan Olsen2. n Engineering background: submarine design
n Stanford MBA
n Product Management leader at Intuit & startups
n CEO & Cofounder, TechCrunch winner YourVersion
n Product Management consultant
n Founder: Lean Product Silicon Valley
Twitter: @danolsen
My slides: https://dan-olsen.com
My Background
Copyright © 2018 @danolsen
4. Product Teams Are Responsible For
Product-Market
Fit
AND Growth
Copyright © 2018 @danolsen
5. n Book giveaway on Twitter
n Tweet: include @danolsen
n Hashtags
#prodmgmt
#leanstartup
#Decisions2018
@SplitSoftware
That’s Why I Wrote
Copyright © 2018 @danolsen
6. Identify highest
ROI idea
Design and
Implement
Analyze How
the Metric
Changes
Brainstorm
Ideas to
Improve Metric
Copyright © 2018 @danolsen
Lean Product Analytics Process
Identify What
Your Metrics Are
Measure Metrics
Baseline Values
Evaluate Metrics
Upside Potential
Global
Level
Metric
Level
Select
Top Metric
Learn
& Iterate
7. At any point, one metric offers the
highest return-on-investment (ROI)
“Metric that Matters Most”
(MTMM)
Focus on right metric at right time
Copyright © 2018 @danolsen
8. If you just launched a new product,
which would you optimize first?
Acquisition: attracting prospects
Conversion: turning prospects into customers
Retention: ensuring customers remain active
Copyright © 2018 @danolsen
9. If you could track only 1 metric to measure
Product-Market Fit, which metric would it be?
Copyright © 2018 @danolsen
10. Retention Rate
n Retention rate tracks what % of your
customers are still active over time
~80%
never use
app again
Curve either
goes to zero
or flattens out
16. Unique Visitors in Last Period x Monthly Retention Rate
Profit = Revenue - Cost
Unique Visitors x Ad Revenue per Visitor
Impressions/Visitor x Effective CPM / 1000
Visits/Visitor x Pageviews/Visit x Impressions/PV
New Visitors + Returning Visitors
Define the Equation of your Business
Peeling the Onion
Advertising Business Model:
Copyright © 2018 @danolsen
18. View Each Metric as a Gauge
Copyright © 2018 @danolsen
Minimum
Possible
Value
Maximum
Possible
Value
Current
Value
19. Identifying Your Metric that Matters Most
n What is the upside potential of each metric?
n How much do we think we can “move the needle”?
n What would the revenue impact be?
n How many resources will it take to move the needle”?
n Developer-days, time, money
n Which metric offers best Return on Investment (ROI)?
Copyright © 2018 @danolsen
20. Copyright © 2018 @danolsen
Prioritizing Product Ideas by ROI
Investment (developer-weeks)
Return(ValueCreated)
Idea C
Idea B
Idea D
Idea A
Idea F
1
1
2 3 4
2
3
4
?
21. Types of Metric ROI SituationsReturn
Investment
Return
Investment
Return
Investment
Metric A
Good ROI
Metric B
Bad ROI
Metric C
Great ROI
Copyright © 2018 @danolsen
23. • Which metric has highest ROI opportunity?
Case Study:
Optimizing Friendster’s Viral Loop
Active
Users
Prospective
Users
Invite Click
Succeed
Invite
click-through rate
Conversion
rate
Don’t
Click
Fail
Invites per
sender
% of users
sending
invites
• Multiplied together, these metrics determine your viral ratio
Users
% of users
who are
active
= 15%
= 2.3
= 85%
Registration
Process
Copyright © 2018 @danolsen
24. The Upside Potential of a Metric
0
100%
0
100%
0
?
Registration
Process Yield
% of users sending
invitations
Avg # of invites
sent per sender
2.3
85%
15%
Max possible
improvement
0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%
Copyright © 2018 @danolsen
26. The Upside Potential of a Metric
0
100%
0
100%
0
?
Registration
Process Yield
% of users sending
invitations
Avg # of invites
sent per sender
2.3
85%
15%
Max possible
improvement
0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%
Copyright © 2018 @danolsen
Metric B
Bad ROI
Metric A
Good ROI
Metric C
Great ROI
27. Okay, so how can we improve the metric?
n How do we increase the average number of
invites being sent out per sender?
n For each idea:
n What’s the expected benefit? (how much will it
improve the metric?)
n What’s the expected cost? (how many engineer-
days will it take?)
n You want to identify highest ROI idea
Copyright © 2018 @danolsen
31. Identify highest
ROI idea
Design and
Implement
Analyze How
the Metric
Changes
Brainstorm
Ideas to
Improve Metric
Copyright © 2018 @danolsen
Lean Product Analytics Process
Identify What
Your Metrics Are
Measure Metrics
Baseline Values
Evaluate Metrics
Upside Potential
Metric
Level
Select
Top Metric
Learn
& Iterate
Global
Level
32. Upping Your Experimentation Game
n Simultaneous tests vs. before-and-after
n Rollout to any % of users vs. only 100%
Feature flags
n Testing multiple variants vs. 1 alternative
n Experimentation platform: buy vs. build
n Experimentation velocity learning velocity
Copyright © 2018 @danolsen
33. I’m teaching a Lean Product
Management Workshop in SF
Oct 24th San Francisco
https://danolsen.eventbrite.com
Save 15% with code SPLIT