← Glossary

Holdout Test

Randomly withholding ads from part of an audience to measure true incrementality

In one line

A holdout test randomly splits users or regions into two groups and withholds ads from one group entirely (the holdout), then compares outcomes against the exposed group.

Why it's needed

Observational data alone can't prove an ad caused a lift in conversions. Comparing a group that saw the ad against a group that genuinely didn't is the only way to confirm real incrementality.

How to interpret it

If the two groups' conversion rates are nearly identical, that's strong evidence the ad had little incremental effect — possibly a sign of cannibalization. A clear gap means the ad is genuinely driving results.

Go deeper

How to design and read a holdout test in practice is covered in Measuring Incrementality.

Related:Incrementality Measurement: Did the Ad Actually Create That Conversion?