There are multiple ways to properly measure the performance of an ad campaign, but there is frequently confusion about the differences between attribution, incrementality and lift (and whether channels measured in different ways can be compared). Getting your measurement right does not require expensive technology, just knowledge and intelligence, and Media Kitchen is here to help.

1. July 31st, 2020
How to measure DR advertising properly

2. 2
IncrementalityAttribution
2
There are two core components to
effective DR campaign measurement

3. There are two core components to
effective DR campaign measurement
3
IncrementalityAttribution
3

4. 4
Complexity
Single touch Fractional /
MTA
Assumes that one
ad made all the
difference,
regardless of how
many ads were seen
Assigns credit to
multiple ads based
on a simple set of
rules
Uses advanced
math / big data and
statistics to assign
credit unevenly
between
touch-points
Heuristics / rule-based models
Multi-touch
(rules based)
A spectrum of different approaches to attribution, with increasing
levels of complexity

5. There are two broad categories: rules-based and fractional
Single touch Multi-touch
(rules based)
Fractional /
MTA
1. First touch /
introducer
2. Last touch
3. Even credit /
linear
4. Time decay
5. Fractional
attribution
Heuristics / rule-based models
5

6. 1. First touch / introducer gives all the credit to the first ad
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
Search
ad click $
Gives all the credit to the first ad
Tends to give more credit to display ads vs. typical
‘closers’ like paid search or SEO
100% 0% 0% 0% 0% 0% 0% 0%
6

7. 2. Last touch assumes that the conversion was due
to the final touchpoint
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
Search
ad click $
Could be:
• Last ad (view-through) – the better option – or
• Last click / interaction (click-through)
0% 0% 0% 0% 0% 0% 0% 100%
7

8. 3. Even credit / linear assumes that every touchpoint was
equally as important
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
Search
ad click $
12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5%
Even credit models are an improvement since they do
not exclusively give credit to one touch, but they
overweight ads that did not introduce the brand or
close the conversion
8

9. 3. Position based (opener / closer) is a slight upgrade, giving the
first and last touches more credit
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
Search
ad click $
20% 10% 10% 10% 10% 10% 10% 20%
Position based over-credits the introducer and closer,
but does not meaningfully add sophistication to the
process
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
9

10. 4. Time decay assumes that more recent ads had more impact on
the purchase decision
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
Search
ad click $
5% 7% 9% 11% 14% 16% 18% 20%
Uses a relatively simplistic methodology to assign
credit on a sliding scale based on the time between ad
exposure and conversion
10

11. 5. Fractional attribution gives different amounts of credit to
each touchpoint
Search
ad click
Remar-
keting
ad
Display
ad
Organic
search
click
Social
media
post
Online
video
Display
ad
Search
ad click $
3% 8% 10% 12% 5% 22% 30% 10%
Uses an algorithm to assign a different amount of
credit to each ad seen by the customer
11

12. Simple Basic Enterprise
Complexity
and cost
Free ~$50k annually >$200k annually
Notes
● Can be set up within days
● Not customizable
● No offline channel
inclusion possible -
non-digital media
measured with
supplemental studies
● Can be setup in 2-4
weeks
● Some customizability is
possible
● inclusion of DM, radio, TV
etc.)
● Can be setup 3-6 months
● Highly customizable in
terms of KPIs etc.
● Can be integrated with
mix models and offline
channels
● Advanced cross-device
Examples
Multiple options exist for attribution solutions for clients
on any budget
12

13. Many of these models are
available today in tools that
many advertisers already
use
13

14. Google’s analytics solutions gallery contains almost 1,000
solutions in the attribution category
14

15. A fractional attribution beta is live in Google Analytics
15

16. ● Some attribution technology will be
affected because there will no longer
be good user level data to model
from
● Walled gardens will be less affected,
but more narrow in the scope of
what they measure
● 1st party based solutions (web
analytics) will continue to operate,
but be far more reliant on clicks than
on view through data
How does Chrome sunsetting cookies affect all of this?
16

17. IncrementalityAttribution
17
There are two core components to
effective DR campaign measurement

18. ● If a user’s journey path only includes a single
touchpoint, any attribution methodology will
attribute all of the credit to that one touch
● It is highly unlikely that this is true - consumers will
always have some other stimulus in their journey,
but in some cases it will not be measurable in
attribution (offline advertising, word of mouth, PR
etc.)
● Incrementality is the impact of advertising over
and above the absence of advertising (the organic
demand that exists thanks to brand strength and
channels like PR that cannot be included in
attribution)
Single touch
Search
ad click $
100%
Search
ad click $
100%
Search
ad click $
100%
Multi-touch
(rules based)
To understand the importance of incrementality, let’s do a thought
experiment - can one search ad drive 100% of a conversion?
Fractional /
MTA
18

19. Lift
The percentage difference between ad-driven results and non ad-driven results
Incrementality
The proportion of your ad-attributed results that are truly driven by advertising
19

20. Ideally, advertisers would be be able to compare incrementality
between channels and partners, but this cannot always be done
Advertising incrementality, by channel ● A view of media
performance that neatly
compares channels to one
another is very difficult to
achieve due to differences
in how incrementality can
be measured:
○ Different measurement
tools and approaches
○ Different metrics
(conversions vs.
conversion rates vs.
convertor rates)
○ Different treatment of
viewable vs.
non-viewable ads
Advertising incrementality, by channel
20

21. ● When modeling lift, the most straightforward approach is to measure conversions, leads or some other outcome
● This works well if your test and control groups are the same side (if your test group is ten times as large as the control group
it wouldn’t be surprising if there are more conversions in the test than the control)
1. And some modeling exercises use events or actions instead of
conversion or convertor rates
Incrementality
Lift
50
Predicted vs. actual leads / conversions / revenue
= 25%
50
200
= 33%
50
150
21

22. Incrementality
Lift
0.5%
Control vs. exposed conversion rates
2. The most simple form of incrementality measures control and
exposed conversion rates
= 25%
0.5%
2.0%
= 33%
0.5%
● A reasonably simple way to measure incrementality is to compare conversion rates of control and exposed populations
● Results should be accurate as long as the control is randomized and there is no bias to the exposed audience
● Conversion rates can be measured using an ad server, fractional attribution platform or any other solution that gives
impression level data (web analytics tools will not work, for instance, since they do not measure impressions)
1.5%
22

23. Control vs. exposed convertor rates
Incrementality
Lift
0.5%
3. The most simple form of incrementality measures control and
exposed conversion rates
= 25%
0.5%
2.0%
= 33%
0.5%
1.5%
● Depending on how the holdout group is set up and what type of attribution is being used to calculate results, it may be
necessary to use convertor rates instead of conversion rates (with users instead of impressions as the denominator)
23

24. ● Some channels and platforms do not allow for the creation of measurable holdout groups (search, OOH, print etc.), which
require the use of modeling to estimate what would have happened in the absence of advertising
● For example, an advertiser may not be able to stop someone from seeing their OOH, but they may be able to model a
control group by not running OOH in certain DMAs, if those DMAs are similar enough to the ones in which OOH is running
Incrementality
Lift
0.5%
Predicted vs. actual conversion rates
4. When it is not possible to hold out users, we may need to create a
synthetic control via modelling
= 25%
0.5%
2.0%
= 33%
0.5%
1.5%
24

25. Incrementality
Lift
50
5. A more complex form strips out non-viewable ad exposure to
compare only exposed users
= 50%
50
100
● An extension to the incrementality calculation methodology is to layer in viewability
● In their standard lift calculations, Facebook exclude non-viewable impressions from their calculations, which increases the
incrementality and conversion rates
● For some media channels (DM, TV etc.) it is not possible to know the non-viewable rate
= 100%
50
50
100
Non-viewable Viewable
Control vs. exposed conversions
25

26. Exposed vs. unexposed conversion rate
How not to measure incrementality...
Comparing data for unexposed and exposed groups of users is tempting and easy, but can
easily lead to misinterpretation
26

27. Exposed vs. unexposed conversion rate
Since there is probably bias in your audiences, you are measuring the
underlying propensity to convert as well as any impact from media
● Age 45+
● Located in Florida
● Income >$100k
● Age 18-34+
● Located in Seattle
● Average income $60k
27

28. Dimension Options
Attribution methodology Single touch, rules based multi-touch, fractional attribution
Analysis methodology Randomized control, synthetic control, predictive model (matched market)
Treatment of non-viewable impressions Included or excluded
KPI / metric Conversion rate, convertor rate or conversions
● Before comparing incrementality rates across channels, make sure that there are not major differences between the
ways in which incrementality was measured, along the following dimensions:
Factors to consider when measuring and comparing
incrementality and lift
28

29. Randomised A/B testing is the gold-standard for incrementality measurement, splitting our audience into
exposed and holdout users
How can we measure the likelihood of conversion in the absence
of ads?
Exposed users see campaign ads Holdout users see a control
29

30. ● They can measure identically sized holdout groups if they want, since there is no actual cost to them of doing so (other
than the opportunity cost of not serving them ads and not receiving revenue from the advertiser)
● Their identity resolution capabilities are (in theory) perfect, since they have logged in data from browsers and apps
across all devices
Social platforms do not need to worry about this, since they can
measure everything on their platform all of the time
30

31. ● Search cannot be tested
in the same way as display
● Google would not allow
an advertiser to show a
United Way ad if
someone searches for
‘Venture snowboards’
Search requires a different methodology
31

32. 32
Shut off date
Time
Nav search
Non-nav search
SEO
There are multiple ways to do this, but here’s one: first, segment
your search results into multiple pieces
32

33. 33
Time
Nav search
Non-nav search
Choose one part of the campaign to shut off (in this case, nav)
SEO
Shut off date
33

34. 34
Actual SEO
Non-nav search
Predicted SEO
Time
Forecast what SEO would have driven with paid still running - then
calculate incrementality using the difference to actual results
Nav search
Shut off date
34

35. Matched market tests can estimate lift comparing conversions
between two very similar test and control geographies
Test
Control
Test Control
Two historically similar
geographic markets
Exposed to
ads
Unexposed to
ads
Using geography to test replaces possible time-based bias with geography-based bias
35

36. ● Media mix models include
an estimate of “baseline”
demand
● This baseline is calculated
against all channels, such
that conversions allocated
to each channel are
inclusive of incrementality
and represent their ‘true’ lift
There are other ways of tackling incrementality, each with their
own specialized focus and methodology
Media mix modeling Propensity modeling Unified analytics
● A type of marketing
measurement platform that
combines econometric
(media mix modeling)
techniques and multi-touch
attribution
● Some providers incorporate
incrementality measures
into the modeling process
● A modeling technique that
estimates for an individual
or audience the likelihood
that they will take a certain
action
● Applied to marketing, this
can represent baseline
demand and be used as an
incrementality estimate
36

37. Attribution Incrementality
Many different ways to do this
and hard to get right
Just as many ways do do this and
even harder to get right
37

38. 1. Don’t worry about having the most sophisticated technology (use
analysis and free, cheap or existing tools when necessary)
2. Do not ignore incrementality - ask your tech partners about it and
be skeptical
3. Make sure that you are optimizing as much as possible to
comparable metrics, all measured in the same way
So…… how should we optimize?
38

39. Will Burghes
Executive Director of Data & Analytics
william.burghes@us.forsman.co
+1 (212) 352-4685
Thank you!
39