The rise of attribution
As marketing increasingly seek to demonstrate their ROI, one piece of the marketing technology stack that often claims to be central to having an efficient marketing program, is attribution modelling.
An attribution model aims to distribute the sale value to any number of touch points that came before a sale, based on the contribution the touch points had, therefore each touch point ends up having a cost as well as a revenue value, making it theoretically easy to work out the amount that should be invested in each channel. Some years ago, marketing attribution became a popular subject that every advertiser wanted to understand. The argument from agencies and technology providers alike to convince advertisers that attribution is an important subject, is very reasonable:
- Most consumers convert after multiple touchpoints
- Since Ecommerce became popular, a lot of touchpoints can be tracked by user level
- We therefore have the data to build a picture of the value of each touchpoint, since the ultimate goal of advertisers is to sell something. This value is best expressed as a proportion of the sale value.
- Each touchpoint will then have a cost as well as a revenue associated with it, so an ROI can be calculated. Given that there is an associated ROI for each channel, type of campaign, even keyword, decisions on where to invest becomes easier and most importantly, accurate.
What is also very convenient, is that most attribution model providers offer sophisticated machine learning algorithms, where all an advertiser needs to do is to pass on the data, the machine will come up with the best model that ‘we can all trust’.
The state of marketing attribution
In a recent article from Econsultancy – the state of marketing attribution – the survey suggests that the top three goals of having an attribution model are:
- Optimising the media mix
- Justifying media spend
- Understanding user journeys
Two questions arise:
- Does attribution help achieve these goals?
- Is attribution necessary for achieving these goals?
User journeys
The third goal – understanding user journeys – certainly does not depend on attribution, this point is well-demonstrated by the existence of attribution vendors who provide attribution technology without any user journeys visualisation tool. We will therefore focus on the first two goals, which effectively can be grouped under the same heading – optimisation.
Optimisation with attributed data
Optimising the media mix and justifying media spend – tend to be the center of the argument for having an attribution model, they are both related to understanding how much revenue, and cost, can be assigned to each channel, hence the channels can be optimised.
In this article, we will show that firstly, the “insight” from attribution modelling, if any, is not in fact that reliable and secondly, the effort in constructing an attribution model – whether this effort is in time, skills or money in building the model – can be diverted to an approach that will lead to better-optimised campaigns, much more in-depth insights about the marketing program the advertiser is running, and perhaps most importantly knowledge that can be shared and built upon.
Attribution modelling in brief
We can all agree that knowing the true value of each channel, is essential for optimisation. The question is therefore whether attribution modelling is the right process to get at the true value, and what does “true value” mean in the first place.
Generally speaking, attribution modelling uses data after the event to derive the magnitude of contribution for every combination of attributes of touch points. For example there may be a coefficient for an affiliate touchpoint, which is a click event, which is at last position of user journey. This coefficient will likely to be different to a coefficient for the combination of affiliate touchpoint, which is a click, but appears at first position of user journey.
The coefficients are derived from statistically comparing user journeys that have a certain attribute vs journeys that do not have the attribute, evaluated usually based on conversion rate, average order value difference. Let’s look at a simple example to illustrate this.
Imagine that the only advertising activity being conducted is display retargeting, the modelling process will compare user journeys that involve retargeting, with those that do not involve retargeting. The difference in conversion rate will be used to construct the attribution model. By this way of evaluating retargeting, retargeting usually shows to have good value – people who have been shown a retargeting ad have higher conversion rate, hence it will have a good positive coefficient in the attribution model.
This concept of comparison is similar to that of medicinal drug trials, where typically a group is separated into control and trial, without the participants knowing which group they belong to, then the trial is conducted where the drug is applied to the trial but not the control group, the difference in cure rate is the difference between applying the drug or not.
There is a very important difference between the retargeting evaluation and the drug trial however: Drug trials are carefully conducted with a random allocation of patients in the trial and control group, to the point that neither the participants nor the health workers know who is in which group. This is done to ensure that there is no bias in the result.
In the retargeting evaluation on the other hand, as marketing practitioners we know that retargeting ads are only served to people who are likely to convert. Comparing users who have been exposed to a retargeting ad and those who have not been exposed, is therefore simply comparing the users who are more likely to convert, with the users who are less likely to convert. It is not rocket science that the people who have been served a retargeting ad would have higher conversion rate, it is self-fulfilling. There is nothing clever statistics can do to “rectify” the problem because the data is inherently biased. This is akin to running drug trials where drugs are only applied to people who are more likely to be cured – this will make pharmaceutical companies very happy as their drugs will surely be shown to work, but will not be very good for science.
Is this problem restricted to retargeting where vendors select the audience before serving them ads?
Unfortunately not, the data fed into the attribution modelling process is “optimised data”, meaning that in many channels, optimisation has already been done to select the audience most likely to convert before serving them any ad. In cases where no ad is served, for example cashback affiliate, users normally only click on a cashback link if they want to claim cashback (i.e. converting) hence cashback tends to have high coefficient in attribution model too.
We have seen that while the attribution modelling process may be robust, if the data used to build the model is biased, the output will also be biased (garbage in, garbage out). The solution requires a change of process and mindset.
Scenario
Let’s look at a real life scenario before we explore a solution.
A car rental company has a large paid search program, they also have an established custom attribution model that tells them how much revenue they are generating through paid search – for every £1 they spend on paid search brand keywords, they are getting £12 back.
Just like any other company, the car rental company is constantly looking at ways to maximise the return from their marketing budget, this includes cutting inefficient spend as well as reallocating budget to new and profitable opportunities.
While paid search brand spend isn’t the largest proportion of their monthly marketing outgoing, very few people believe that it is delivering a true incremental return of 12:1 – in other words, if they switch off PPC brand, few people would expect that revenue would be reduced by the amount reported by the attribution model. No one however knows to what extent PPC brand spend is incremental.
The only way to figure it out is by testing.
The details of how such a test is conducted is not covered in this article, but the result from the test shows that the average cpc can be dropped by half without affecting revenue, but if CPC is reduced further then revenue will begin to suffer.
The action is clear. There are immediate PPC brand efficiency savings to be realised, but the learning from the test has a wider implication. By doing a control test, we are able to gather data and information that no attribution modelling can give us, and we have the confidence that the data collected from the test is sound because we have taken care to design the test so that it is unbiased.
The “test, optimise, model” approach is a change of process and mindset, emphasising the importance of testing, which in turn drives optimisation. It diminishes the role of attribution modelling because the insight from tests always trumps the insight from attribution modelling.
How to implement this change of mindset in practice?
Let’s see how attribution-led optimisation compares with a test-driven approach first.
| Attribution-led optimisation | Test-driven optimisation | |
| Statistical skills | Likely to outsource the modelling skills to a 3rd party vendor. | The statistical skills are relatively easy to acquire for testing purposes. |
| Implementation | Tags are usually required to be put on site to track marketing activities for revenue to be attributed to. | Tags are required to track marketing activities, for some tests, tags have to be manipulated to define audience segmentation. |
| Once orders are attributed by the 3rd party vendor, using the numbers is no different to other models such as last click. | For every test, KPIs could be slightly different and have to be pre-defined, interpreting the test result and subsequently drawing conclusions from them require careful consideration and experience. | |
| Optimisation accuracy | The soundness of the modelling methodology depends on the vendor. However, the model will be as biased as the historical data used to build the model. | All tests begin by designing how unbiased data can be collected, tests are as reliable as the designers of the tests can make them. |
| Control | Most attribution model vendors offer a “black box” solution, advertisers often only receive the output of the process without being able to gain granular insight as to how and what elements of marketing channels work. | Advertisers have complete control over the granularity of marketing that they wish to understand, tests can be tailored to answer most questions advertisers have. |
| Learning and sharing | Other than high level learnings such as how attributed revenue compares with other models, learnings are not transferable. | Learnings from each test can be applied to future campaigns or even other regional marketing teams in the company, as long as the conditions of the test and future / new environments are similar. |
At VISU.AL , our missions is to empower advertisers by bringing rigour to the optimisation process, to increase confidence in the data and knowledge advertisers collect by creating unbiased data through control testing. Our objective is to give you, the advertisers, control over what you can learn from your data, giving you the confidence to evolve your marketing campaigns though rigorous testing, and avoid the unnecessary work and waste of resources that does not further the purpose of optimisation.
Solution- Deliverables
- To reduce inefficiencies by optimising to incremental revenue
- To give confidence in using data and knowledge collected by the advertiser, by bringing rigor to the optimisation process.
- To enable knowledge sharing by systematically recording validated test results.
Process outline
Discovery
- Understand client’s KPIs, campaign structure, past campaign performance.
- Understand client’s business cycles, traffic and conversion volume, which will help with test designs.
- Agree with client the priority of testing, usually based on size of channel and potential gain.
Test
- Agree with client on the test plan for the first element of marketing to be tested, how the test will be conducted and evaluated.
- Advise client on how the test can be implemented using client’s preferred technology, ensuring that the data collected from the test will be free of inherent bias.
- Monitor the test and the data collected during the test.
Evaluation
- Based on the test plan, evaluate the test
- Provide interpretation of the test result, highlight important assumptions that might have been made and conditions under which the test result can be applied.
- Record the test result in a easily accessible and sharable format, facilitating knowledge sharing, based on client’s requirements.
Recommendation
- Recommend best course of action based on the test result.
- Suggest the next test to be conducted, repeat the test cycle so as to build up a knowledge base of the client’s marketing program.
If you would like to understand how we can improve your results through our ‘TOM’ framework , please contact us in the first instance to set up an initial consultation.
