Incrementality Testing: Which Orders Are Truly New?
Attributed orders do not always mean new business. Compare a control group, log contamination, and read business results before keeping, reducing, or retesting spend.
Author
Ranfeng WeiPublished
Updated
Last reviewed
Review scope This lesson maintains 3 linked references; recheck current platform, account, and market details before acting.
Public preview
Understand what this lesson solves
Attributed orders do not always mean new business. Compare a control group, log contamination, and read business results before keeping, reducing, or retesting spend.
Learn to read CPM, CPC, CTR, CPA, ROAS, and attribution as one decision system, so you can tell whether to fix, pause, or scale spend.
Lesson outline
- 1Write whether the order was newly created by ads
- 2Decide whether a holdout is workable now
- 3Read platform numbers beside Shopify business numbers
- 4Write the incrementality review
Public core framework
- Turn the question into plain language: would orders, new customers, and contribution profit fall without this ad layer? Before reading platform ROAS, name the layer being tested: branded search, remarketing, PMax, audience, or region.
- Check order volume, whether regions or audiences can be separated, whether promos, stock, and email cadence are stable, whether the buying cycle is covered, whether tracking is stable, and whether a 10-14 day test is safe enough.
- Compare platform revenue with Shopify total orders, new customers, non-paid orders, AOV, refunds, checkout completion, and contribution profit. If platform revenue falls but total orders and profit hold, the layer may be capturing existing demand.
Sources and review
Last checked: 2026-10-02- arxiv.orghttps://arxiv.org/abs/2604.26169
- support.google.comhttps://support.google.com/analytics/answer/10596866
- arxiv.orghttps://arxiv.org/abs/2602.05997
Applies to: This preview is for the lesson's public problem, structure, and operating boundary. Verify current platform, account, and business facts separately.
Limitations: The preview is a bounded summary and excludes member text, personalized interaction data, and full case derivations; recheck source pages before acting.
Reviewed by: Ranfeng Wei · Next review: 2027-03-31
Checking membership access for this account.
Share this lesson with your reviewer
Share it with the copyable lesson notes so everyone reviews the same evidence, decision line, and next action.