Attribution done right to maximize incrementality

1. Attribution done right to
Maximize Incrementality
Tuomas Mäki
tuomas.maki@smartly.io
Data Scientist

2. Why this training?
Major impact on real results

3. Attribution is a way to measure
Marketing Performance

4. A complex model used wrong can
hurt performance
What is the problem we are
trying to solve?

5. “Half the money I spend on advertising is wasted;
the trouble is, I don’t know which half"
-John Wanamaker

6. The challenge
Did not buy Bought
BUT would have bought anyways
Bought
because saw an ad
Incremental revenue

7. Attribution model should
Mimic Incremental Revenue

8. Additional complexities:
Lifetime Value
Network Effect
Late Conversions
Even more complex:
Organization might value chosen attribution model
over incrementality

9. All attribution models are wrong

10. A simple model used right is very
helpful

11. Contents
What is an attribution model used for?
Types of attribution models
So what should I do?

12. Bid and Budget
Overall investment level
Interpret test results
Employee bonuses

13. Bid and Budget
Overall investment level
Interpret test results
Employee bonuses

14. Campaign A Campaign B
140 %
Vendor Attribution
Campaign A Campaign B
Attribution that
mimics
incrementality140 %
130 %
75 %
Suggests identical performance Reveals a difference
Theoretical Budget Allocation Example
Return on investment

15. Contents
What is an attribution model used for?
Types of attribution models
So what should I do?

16. Econometric models
Vendor attribution models
Rule-based models
Algorithmic models

17. Econometric models
Vendor attribution models
Rule-based models
Algorithmic models

18. Econometric models OR Marketing Mix Modeling
(MMM)
Simplified
Top-down
Increase in sales
= β0 + β1* sales investment + β2* seasonality + β3* TV spend + β4 * FB spend + ...
Often complements ‘bottom-up’ approaches

19. Econometric models
Vendor attribution models
Rule-based models
Algorithmic models

20. Vendor attribution models
E.g., Facebook 7d click + 1d view – All vendors have
similar attribution models
No de-duplication across channels
Assumes all conversions are incremental

21. How to choose attribution window
for vendor-based models?

22. –> Attribution model ≈ 1 d click + 1 d view
Who was affected by ads?
Time-to-conversion

23. Challenges with Time-to-conversion method
It’s more complex than that
We might have just made the decision faster
The “Retargeting hammer effect” – If we reach every
customer every day, whenever they purchase, they
end up being attributed to the first day after
click/impression

24. Econometric models
Vendor attribution models
Rule-based models
Algorithmic models

25. Rule-based attribution models

26. Rule-based attribution models

28. Last click model
Still most common
Over-attributes search and retargeting
Blind use = wasting money

29. Other rule-based models
Best to understand results from all perspectives
Most models (other than first and last click) give very
similar results on aggregate
Still assume all conversions are incremental

30. How to deal with model
imperfections?

31. Check results with lift studies

32. ”The Haircut Method”
Lift study
Attributed-To-Incremental conversions ratio
Separate per channel and funnel step

33. Challenges with the Haircut method
Incrementality varies based on spend-level
Incrementality varies across markets
Incrementality varies over time
Not possible with all channels
Short vs. Long term impact
Incrementality is often very low

34. Task 1
A customer is looking at both FB and Google Analytics attribution
There is a large discrepancy in the numbers.
Why does this happen?
Which one should they trust?

35. Task 2
Can I put prospecting and retargeting campaigns in the same
budget pool?

36. A Lift Study: prospecting vs retargeting (results)
Prospecting
Attributed CPA: 135 USD
75 % Lift
Retargeting
Attributed CPA: 116 USD
0 % Lift
CPA was higher but the lift was better:
Major impact on real results.
Additional complexities:
Lifetime Value
Network Effect
Late Conversions

37. Econometric models
Vendor attribution models
Rule-based models
Algorithmic models

38. Algorithmic attribution models

39. Sidenote:
Mobile Measurement Partners
≠
Multi-Touch Attribution solutions

40. Algorithmic attribution models
Complex Machine Learning models on individual
customer journey level

41. Path-based modeling
Include/Remove a touchpoint + measure impact
KPI
KPINo Ad

42. Challenges with path-based modeling
Not possible to force customer journeys
Hard to find similar paths with long journeys
Missing data
Intent-to-treat*
* https://www.kellogg.northwestern.edu/faculty/gordon_b/files/kellogg_fb_whitepaper.pdf

43. Customer Response Models
Simplified
Probability
to convert
Demography
Market
Interaction history
Attribute based on change
in conversion rate
Or a ‘Survival Model’
= “How long will it take before a conversion
happens”

44. Challenges with Regression Adjusted
Models
Modeling accuracy unclear
Complex and expensive
Black box
Missing data

45. Typical implementation project for an
algorithmic model
3 – 12 months*
Requires a close-to full-time project manager*
Involve at least marketing, finance, IT
Cost: high
Sweet spot: Annual cross-channel spend > 20 MUSD, at least 3 – 4 different
channels in use
* Gartner MTA report

46. Contents
What is an attribution model used for?
Types of attribution models
So what should I do?

47. What should I do?
Know the assumptions and limitations behind models used
Compare results within different attribution models
Cross-check results with lift studies
Decisions rather based on hunch than flawed data
A simple model used right is already very useful

48. ”Incrementality-calibrated
algorithmic MTA is the holy grail of
Attribution”
But not worth the investment for everyone

49. Econometric models
Vendor attribution models
Rule-based models
Algorithmic models
Vendor or rule-based (other than last click)
with incrementality calibration for most
Incrementality-calibrated algorithmic MTA for
the most advanced

50. Read more (or use Google):
“Kellogg Whitepaper” – Why measuring incrementality is difficult without a Randomized Controlled
Trial
Gartner Report for Multi-Touch Attribution Market
“The Good, The Bad, and The Ugly blog post”
Read more about Econometric Approach to attribution