Building Analytics for Growth

© 2018 Autodesk, Inc.
Grow Camp 2018
Building Analytics for Growth
Kareem Azees
Digital Marketing & Analytics | @kareemazees | kareemazees.com

Who here has heard about
experimentation in Grow
Camp so far?

Build -> Measure -> Learn
What you have learned so far
1. BRAINSTORM 2. PRIORITIZE 3. TEST
4. IMPLEMENT
5. ANALYZE
6. SYSTEMIZE

Build -> Measure -> Learn
Why you still can’t act on it
1. BRAINSTORM 2. PRIORITIZE 3. TEST
4. IMPLEMENT
5. ANALYZE
6. SYSTEMIZE
No clear KPIs
Poor tracking
Bad instrumentation
Wrong conclusions

Without correct
instrumentation of data
analytics… experimentation
is all a lie.

Data can tell you any story
with some creativity (and
manipulation).

Agenda
What I’ll be covering today
Mindset
What makes a good metric
Metrics related to your business
Toolset
Building data infrastructure
Recap / Lunch
Skillset
In-depth examples
Refining your metrics

Who am I?
A tidbit of my work
Monetization
Referral programs
Customer development
Product analytics
Data infrastructure
Metrics & reporting
Web analytics
Channels and campaigns
Conversion rate optimization

Agenda
Mindset
Toolset Building data infrastructure
Recap / Lunch
Skillset
Analyze your business
Find the metrics that matter

To lead into the workshop, can
you take 2 minutes now to jot
down a few metrics you are
tracking currently?

Qualitative vs. Quantitative
Types of metrics
Qualitative
• Observable
• Behaviours
• Quality
• Surveys, customer
interviews, etc.
Quantitative
• Measurable
• Numbers and stats
• Quantity
• Tracking usage, clicks,
etc.

Example: Survey vs. button clicks
Types of metrics
What people thought about the website. How many clicks did a button get?

Vanity vs. Actionable
Types of metrics
Vanity
Numbers or stats that look
good on paper, but don’t
really mean anything
important.
Actionable
Stats that tie to specific and
repeatable tasks you can
improve and to the goals of
your business.

Example: Total users vs. users registered per month
Types of metrics
0
100
200
300
400
500
600
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Total Users
0
20
40
60
80
100
120
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Net New Users Per Month

Lagging vs. Leading
Types of metrics
Lagging
Historic metric that shows
how you’re doing
Leading
Number today that shows a
metric tomorrow.

Example: Sales results vs. new leads
Types of metrics
Sales results for the past quarter.
New sales leads can help predict
revenue for the quarter.

Correlation vs. Causation
Types of metrics
Correlation
Statistical measure (a
number) that describes the
size and direction of a
relationship between two or
more variables.
Causal
Indicates that one event is
the result of the occurrence
of the other event.

Correlation ≠ Causation
Types of metrics

Output vs. Outcome
Types of metrics
Output
Tells the story of what you
produced or your
organization's activities.
Outcome
The level of performance or
achievement that occurred
because of the activity or
services your organization
provided.

Example: 10 blog posts vs. +30% time on site
Types of metrics
Write 10 blog posts Increase time on site by +30%

What makes a good metric?
§ Understandable: What does it mean?
§ Comparative: Against time, users, etc.
§ Ratio/Rate: X variable / Y variable
§ Behaviour Changing: Does this invoke change?
Understanding metrics

Example: Free Trial Conversion
§ Understandable: How many users converted from free
trial to paid per week
§ Comparative: 10% free trial to paid conversion vs. 10%
free trial to paid conversion per week
§ Ratio/Rate: Total users who completed free trial vs.
converted customers / free trials started
§ Behaviour Changing: It is low, so we need to test to
improve the trial experience
Understanding metrics

Agenda
Mindset
Recap / Lunch
Skillset
In-depth examples
Refining your metrics

Metrics depend on your business type
But they generally are similar: CPA, CAC, LTV, ARPU, Churn
Source: Lean Analytics

SaaS Product Example
Acquisition
Channel Traffic Conversion
CAC, CPA by persona
CAC, CPA by channel
Multi-touch
attribution
Landing Page
Optimization
Real time
personalization
Checkout flow
conversion
Drop-off paths
Pricing elasticity test

CPA vs. CAC
Acquisition
Cost Per Acquisition
Measures the cost to acquire
something that is not a
customer — for example, a
registration, activated user,
trial, or a lead
Customer Acquisition
Cost
Measures the cost to acquire
a customer.

Factors Included in CAC
Acquisition
§ Tools, overhead, salaries are all marketing expenses
§ Measure over time, by channel, persona, etc.
§ Non fully loaded CAC = understand channel performance
§ Fully loaded CAC = understand holistic marketing performance
!"! =
$%&'()*+, + .%/(. (01(+.(.
# 34 56.)37(&. %586*&(9

Analyze Channel Performance Alone
Display Campaign Example
Metric Value
Date Feb 1-7
Campaign Name Campaign-1
Spend $10,000
Impressions 4,000,000
Clicks 1,500
Subscriptions 32
Billings $16,000
ROAS ($16,000/$10,000)*100 = 60%
CAC ($10,000/32) = $312.50

View-through vs. Click-through
Acquisition
View-through
conversion
A visitor is shown your ad,
they do not click on it, but
later return to your site and
convert.
Click-through
conversion
A visitor is shown your ad,
they click on it, and then
convert.

Autodesk spends $ on display, affiliate,
and SEM with all owners reporting
positive and growing return on ad
spend (ROAS). But new seats are
falling. How can this be the case?

Using data to check view-through value
§ Run an experiment with a charity ad and same ad spend
§ Incremental impact that view-through has is +17%
Example display experiment
Test Cell Spend Orders via
Display
Orders that also
touched SEO/SEM
Campaign-1 $1,000 100 70
Red Cross
Campaign
$1,000 50 50 +17%
+20 orders

Understand Multi-Channel Performance
Many channels, many touchpoints
Double counting spend is a difficult problem

Activation
Onboarding Activation
Magic Moment
Time to complete
Notifications
Email Strategy

Onboarding
Activation
Understand how to onboard customers and how to personalize by persona.

Test to Improve User Experiences
Activation
Analyze correlation data to find causations that
improve first user experiences.
Source: Amplitude

Examples of Activation Strategies
Activation
DropBox: Your first file is backed up from your computer into the
cloud
Facebook: You connect with 7 friends within your first 10 days
Stack Overflow: Your question is answered
Instacart: When your groceries are delivered for the first time
Instagram: Someone likes one of your photos

Retention
Usage Retention Financial Retention
Churn
Retention
Time in app
Involuntary churn
Voluntary churn
LTV

If someone isn’t using your product, they will churn
Usage-based retention
Source: Amplitude

Cohort analysis key to retention
Transaction based retention
Source: Christoph Janz
Aggregated retention rates for high level reporting. By cohort for improving it.
0
200
400
600
800
1,000
1,200
Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13 Aug-13 Sep-13 Oct-13
Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13 Aug-13 Sep-13 Oct-13

Customer retention chart
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
J
a
n
-
1
3
F
e
b
-
1
3
M
a
r
-
1
3
A
p
r
-
1
3
M
a
y
-
1
3
J
u
n
-
1
3
J
u
l
-
1
3
A
u
g
-
1
3
S
e
p
-
1
3
O
c
t
-
1
3
Customer Retention Chart

Use this data to build LTV models
There are many ways to calculate LTV. Fully loaded or non fully loaded.
Source: Christoph Janz
Where:
• ARPU = average revenue per user
• Avg. Cust. Lifetime, n = the inverse of the churn, n=1/annual churn
• WACC = weighted average cost of capital
• Costs = annual costs to support the user in a given period, cost of goods sold
• CAC = customer acquisition costs
!"# = %
&'(
)
*+,-& − /0121&
(1 + 6*//)&
− /*/

Is it possible to have a
negative customer LTV?

Yes, MoviePass would have negative LTV
§ Non fully loaded CAC: $60
§ ARPU (per month): $10.50
§ Churn (per month): 4%
§ Gross Margin: 18%
§ Annual Discount Rate: 10%
§ COGS (per movie): $8.61
Source: Tren Griffin

$-20.90 LTV and even higher if movie consumption >1/month
Source: Tren Griffin

A company loses a quarter
of its customers every year.
Is this good or bad?

Baselines
No. Hotjar has 2.5% net churn m/m.
Measure it, then find out if you are out of norm.

Baselines
§ Pixlr was monetizing
via ads so spent bulk
of time on retention
§ Understood we were
just in line with
retention for photo &
video apps
Product Retention Rates
Source: Flurry Benchmark

Baselines
Product Usage
Source: Mixpanel Benchmark Report 2017

Baselines
Web Conversion Rates
Source: Mixpanel Benchmark Report 2017

Baselines
Other Metrics
Source: Lean Analytics
Churn
2%/month
Free to paid
2% of free users
Page load time
<5 seconds
LTV: CAC
3:1

Getlatka.com - tons of more benchmarks
Baselines
Source: GetLatka.com
Churn
2%/month
Free to paid
2% of free users
Page load time
<5 seconds
LTV: CAC
3:1

Data Infrastructure Options
How do you decide what to use?

Founding Stage (0 to 20 employees)
Data Infrastructure
Track only the metrics that move your business
forward. After your first marketing hire, use
UTM tracking.
Web tracking
Product tracking
Financial Reporting

Mid Stage (20 to 100 employees)
Data Infrastructure
Start forming the first data pipeline. Hire first
analytics lead.
ETL
Collect Load & Operate Transform & Analyze
Enrich

Extract, Transform, and Load
Data Infrastructure
Do not build in-house ETL. As employees leave, becomes really
difficult and costly to maintain.

Late Stage (100+ employees)
Data Infrastructure
Migrate existing event/web tracking tools to
Snowplow due to poor scaling costs. Extract data
from marketing tools to data warehouse. Document.
Collect Load & Operate Transform & Analyze
Enrich
ETL

Marketing Technology
Data Infrastructure
Test channels and track reporting. Afterwards,
figure out the best tool for the job.

Summary
Recap
1. Metrics should be understandable,
comparative, a ratio, and behaviour changing.
2. Find metrics that matter to your business,
measure them, slice and dice them, and
compare them to a baseline.
3. Start using data early and add more capability
as you scale.

© 2018 Autodesk, Inc.
Grow Camp 2018
Building Analytics for Growth - Workshop
Kareem Azees
Digital Marketing & Analytics | @kareemazees | kareemazees.com

Agenda
Mindset
Recap / Lunch
Skillset
In-depth examples (20-30 min)
Refining your metrics (remainder)

Buffer’s Baremetrics Dashboard
https://buffer.baremetrics.com/
Source: Buffer

Discuss the performance of this
company by the metrics
provided.
https://autodesk.box.com/v/grow-camp

Litmus test of any company’s financial
success is the ability to acquire many
high lifetime value (LTV) customers.
Being LTV-centric is at the heart of
being customer centric.
Daniel McCarthy, Asst. Professor, Emory University
Source: Daniel McCarthy

72% of customers churn by month six
100% - 28% = 72%
45%
38%
34%
31%
30%
28%
27% 26% 25% 25% 24% 23%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
1 2 3 4 5 6 7 8 9 10 11 12
Percent
of
Cohort
Still
Active
Number of Months Since Acquisition
Subscriber Retention by Number of Months Since Acquisition

Market saturation shown by increasing CAC
9.5% growth in CAC from Feb to Mar 2017
$105
$92
$82
$73
$68
$64
$56
$52
$48
$56
$58
$61
$61
$63
$65
$64
$66
$67
$63
$62
$61
$72
$78
$84
$85
$90
$94
$105
$115
$124
$114
$113
$111
$134
$147
$161
$-
$20
$40
$60
$80
$100
$120
$140
$160
$180
A
p
r
2
0
1
4
M
a
y
2
0
1
4
J
u
n
2
0
1
4
J
u
l
2
0
1
4
A
u
g
2
0
1
4
S
e
p
2
0
1
4
O
c
t
2
0
1
4
N
o
v
2
0
1
4
D
e
c
2
0
1
4
J
a
n
2
0
1
5
F
e
b
2
0
1
5
M
a
r
2
0
1
5
A
p
r
2
0
1
5
M
a
y
2
0
1
5
J
u
n
2
0
1
5
J
u
l
2
0
1
5
A
u
g
2
0
1
5
S
e
p
2
0
1
5
O
c
t
2
0
1
5
N
o
v
2
0
1
5
D
e
c
2
0
1
5
J
a
n
2
0
1
6
F
e
b
2
0
1
6
M
a
r
2
0
1
6
A
p
r
2
0
1
6
M
a
y
2
0
1
6
J
u
n
2
0
1
6
J
u
l
2
0
1
6
A
u
g
2
0
1
6
S
e
p
2
0
1
6
O
c
t
2
0
1
6
N
o
v
2
0
1
6
D
e
c
2
0
1
6
J
a
n
2
0
1
7
F
e
b
2
0
1
7
M
a
r
2
0
1
7
Estimated Marketing Spend Per Customer Acquisition

As customers become more loyal, they spend less
30% less monthly revenue per active customer over seven month span
$147
$124
$120
$115 $113 $109
$104
$-
$20.00
$40.00
$60.00
$80.00
$100.00
$120.00
$140.00
$160.00
$180.00
$200.00
Sep 2016 Oct 2016 Nov 2016 Dec 2016 Jan 2017 Feb 2017 Mar 2017
Expected Monthly Revenue Per Active Customer
September 2016 Acquisition Cohort

LTV has decreased by 20% over two year span
There is only so much the most loyal customers can spend
$997
$977
$956
$936
$925
$914
$905
$895
$885
$873
$855
$837
$911
$889
$868
$846
$833
$821
$812
$830
$848
$864
$873
$881
$808
$789
$770
$750
$740
$729
$721
$724
$728
$729
$-
$200.00
$400.00
$600.00
$800.00
$1,000.00
$1,200.00
J
a
n
2
0
1
4
F
e
b
2
0
1
4
M
a
r
2
0
1
4
A
p
r
2
0
1
4
M
a
y
2
0
1
4
J
u
n
2
0
1
4
J
u
l
2
0
1
4
A
u
g
2
0
1
4
S
e
p
2
0
1
4
O
c
t
2
0
1
4
N
o
v
2
0
1
4
D
e
c
2
0
1
4
J
a
n
2
0
1
5
F
e
b
2
0
1
5
M
a
r
2
0
1
5
A
p
r
2
0
1
5
M
a
y
2
0
1
5
J
u
n
2
0
1
5
J
u
l
2
0
1
5
A
u
g
2
0
1
5
S
e
p
2
0
1
5
O
c
t
2
0
1
5
N
o
v
2
0
1
5
D
e
c
2
0
1
5
J
a
n
2
0
1
6
F
e
b
2
0
1
6
M
a
r
2
0
1
6
A
p
r
2
0
1
6
M
a
y
2
0
1
6
J
u
n
2
0
1
6
J
u
l
2
0
1
6
A
u
g
2
0
1
6
S
e
p
2
0
1
6
O
c
t
2
0
1
6
Acquisition cohort
Cumulative Revenue After 6 Months Per Active Customer by
Acquisition Cohort

CAC: LTV Analysis
§ No COGS included in six month LTV
§ If variable contribution margin = 26% and CAC = $161, then $619 in net revenue
§ Net revenue = gross revenue minus returns and promotional discounts)
§ But wait… $619 in net revenue comes in five months
§ Correct, but remember only 30% of customers were around after 5 months
§ So on the remaining 70%, negative LTV:CAC
§ At $161 per customer, only 30% of customers actually are profitable
§ Break even moves further away as revenue per customer declines and CAC rises
$729: $114 = 6.39x ROI (Oct 2016)

Pull back out that list of
metrics you made earlier. What
stage are you in, what metric is
it, and would you change it? If
yes, how?

Autodesk and the Autodesk logo are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their
respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document.
© 2018 Autodesk. All rights reserved.
Thank you.
@kareemazees
kareem.azees@autodesk.com
kareemazees.com

Building Analytics for Growth

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Building Analytics for Growth

Ähnlich wie Building Analytics for Growth (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Building Analytics for Growth