What is Incrementality Testing in Marketing?

Attribution models answer the question "which touchpoints did the converting customer interact with?" Incrementality testing answers a fundamentally different question: "would this customer have converted anyway, even without the marketing?" The distinction matters enormously for budget allocation.

A retargeting campaign might show a 10x ROAS in your attribution dashboard. But if 80% of those customers would have purchased without seeing the ad, the true incremental ROAS is 2x. Without incrementality testing, you cannot tell the difference.

The Attribution Gap

Why Attribution Overstates Impact

Attribution gives credit to touchpoints that were present in the conversion path, but presence does not prove causation. Brand search ads appear in almost every conversion path because customers search your brand name before buying. But they were going to search and buy regardless - the ad did not cause the purchase, it intercepted it.

Similarly, retargeting shows ads to people who already visited your site and showed purchase intent. High conversion rates from retargeting are partly real influence and partly preaching to the choir.

What Incrementality Reveals

Incrementality measures the lift - the additional conversions that happened because of the marketing, minus the conversions that would have happened without it. If 100 people in your ad group convert and you estimate 70 would have converted anyway, the incremental lift is 30 conversions. That is the true value of the campaign.

How Incrementality Testing Works

The Holdout Method

The gold standard is a randomized controlled experiment. Split your target audience into two groups: a treatment group that sees the ad and a holdout group that does not. Compare conversion rates between the two groups. The difference is the incremental lift.

For example, expose 90% of your Facebook retargeting audience to ads (treatment) and suppress ads for 10% (holdout). If the treatment group converts at 5% and the holdout converts at 3.5%, the incremental lift is 1.5 percentage points. The campaign caused 30% of its attributed conversions - the other 70% would have happened anyway.

Geographic Testing

When audience-level holdouts are not possible, geographic testing compares regions. Run the campaign in some markets and not others, then compare outcomes. This works well for brand campaigns, TV, and out-of-home advertising where individual-level targeting is difficult.

Time-Based Testing

Turn campaigns on and off in alternating periods and measure the impact on conversions. This is simpler to execute but less precise because external factors (seasonality, competitor activity) can confound the results.

Running Your First Test

Choose the Right Campaign

Start with your largest or most expensive campaign - the one where incorrect attribution has the biggest financial impact. Retargeting campaigns are excellent first candidates because they typically show high attributed ROAS but often have significant organic overlap.

Design the Test

Define your success metric (usually incremental conversions or incremental revenue), determine the holdout size (5-15% of the audience is typical), and calculate the test duration needed for statistical significance. Most tests need 2-4 weeks to produce reliable results.

Measure Results

Compare conversion rates between treatment and holdout. Calculate the incremental lift and incremental ROAS. If the campaign's incremental ROAS is still positive, keep running it. If it is negative - meaning you are paying more for the ads than the incremental revenue they generate - reduce spend or reallocate to channels with proven incrementality.

Incrementality by Channel

Channels with High Incrementality

Prospecting campaigns targeting new audiences typically show high incrementality because the customers had no prior relationship with your brand. Content marketing and SEO drive organic discovery that would not happen without the investment. Influencer partnerships introduce your brand to new audiences.

Channels with Low Incrementality

Brand search ads often have low incrementality because customers would find your site through organic search. Retargeting frequently has lower incrementality than attributed ROAS suggests. Loyalty emails to active customers may trigger purchases that were coming anyway.

The Budget Implication

Understanding incrementality often shifts budget from bottom-funnel channels (retargeting, brand search) toward top-funnel channels (prospecting, content) that create genuinely new demand. This feels counterintuitive because top-funnel campaigns show worse attributed performance, but they drive more incremental revenue.

Combining with Attribution

Incrementality testing and data-driven attribution are complementary. Attribution tells you how to distribute credit across touchpoints in real time. Incrementality testing calibrates attribution by revealing which channels are over-credited and which are under-credited.

Use incrementality results to adjust your attribution model weights. If testing shows retargeting has 40% incrementality, reduce retargeting's attribution credit by 60% to get a more accurate picture of its true contribution.

Arktis provides multi-touch attribution that can be calibrated with incrementality insights, giving you both real-time attribution and validated causal measurement.

Sources

1. Meta, "Conversion Lift Studies"

2. Google, "Brand Lift and Conversion Lift"

3. Harvard Business Review, "Measuring True Marketing Impact"