Retail Media Measurement: Clean Rooms, Incrementality and Closed-Loop Attribution

Every retail media network will tell you its ads are working. Amazon reports a healthy ROAS, Walmart Connect reports another, Instacart or a regional network reports a third, and the totals add up to more sales than your finance team can find. None of these reports is necessarily wrong. They are simply answering a narrower question than the one you care about: would those sales have happened without the ads?

In short: good retail media measurement combines three layers. Platform reporting, including closed-loop attribution and new-to-brand metrics, guides daily optimization. Clean rooms such as Amazon Marketing Cloud let you join ad exposure with deeper data in a privacy-safe way. Incrementality tests, such as holdouts and geo tests, show how many sales the ads actually caused. Budget decisions should rest on the third layer, not the first.

1ROAS vs Incrementality

Return on ad spend divides attributed sales by ad spend. It is quick to calculate and useful for comparing campaigns within one network, but it credits the ad for every attributed sale, including sales from shoppers who were going to buy your product anyway. Branded search is the classic example: shoppers who search for your brand name often convert well, so ROAS looks excellent, yet many of them would have found you without the ad.

The two measures answer different questions:

High ROAS with low incrementality usually means you are paying for sales you already had. Lower ROAS with strong incrementality can be the better investment, particularly for prospecting and new-to-brand campaigns.

2Closed-Loop Attribution and Attribution Windows

Retail media's main advantage over most other channels is closed-loop attribution. The retailer sees both the ad exposure and the purchase, so it can connect them directly instead of relying on modelled conversions. That is valuable, but it comes with limits worth understanding:

• Each network uses its own rules. Attribution windows, the period after a click or view during which a purchase is credited, differ by network and sometimes by ad type. Some count view-through conversions, others only clicks.
• Halo sales are counted differently. Some networks credit purchases of other products from the same brand, not just the advertised SKU. That can be legitimate, but it inflates comparisons with networks that don't.
• Networks only see their own store. An ad on one retailer that leads to a purchase on another, or in a physical shop, is usually invisible to the network that ran it.

The practical rule is simple: never add up ROAS across networks or compare it side by side without first writing down each network's attribution window and counting rules. Often the fairest comparison is the one you build yourself, using a consistent definition across networks.

3New-to-Brand Metrics

New-to-brand metrics, offered by Amazon Ads and some other networks, show how many attributed purchases came from shoppers who hadn't bought from your brand within a set lookback period. They are one of the most useful signals in retail media because they separate growth from repeat purchase.

Use them to judge upper-funnel and prospecting activity, such as display, video and offsite campaigns, where ROAS alone tends to undervalue the work. Keep in mind that the definition of "new" depends on the network's lookback period and only covers purchases on that retailer, so a shopper who buys you regularly elsewhere may still count as new-to-brand. Treat these metrics as directional rather than an exact count of new customers.

4Clean Rooms: Amazon Marketing Cloud and Retailer Clean Rooms

A clean room is a secure environment where an advertiser's data and a retailer's data can be analysed together without either side seeing the other's raw, user-level records. Amazon Marketing Cloud is the best-known example in retail media; other retailers and data partners offer their own clean rooms or work through third-party clean room technology.

In general terms, clean rooms work like this:

• Privacy-safe joins: your data, such as CRM or first-party purchase lists, is matched with the retailer's ad exposure and purchase data using protected identifiers, inside the clean room.
• Aggregation thresholds: results only come out as aggregated figures, and queries that would describe too few people are suppressed or blocked. This protects individuals but means very narrow segments may return nothing.
• Query-based analysis: analysts write queries, often in SQL or through templates, to answer specific questions rather than browsing a dashboard.

Typical uses include path-to-purchase analysis across ad types, overlap between campaigns, frequency analysis, reach of new audiences, and building custom audiences for activation. Clean rooms need clear questions, clean input data and people who can write and check queries, so they suit brands with meaningful spend and a measurement plan. Our clean room setup and attribution service handles that setup, CRM matching and analysis.

5Incrementality Testing: Holdouts and Geo Tests

Incrementality testing compares a group that saw your ads with a similar group that didn't. The difference in sales between the two is the lift the ads caused. The two most common designs compare as follows:

A few rules keep tests trustworthy. Decide the question and the success metric before the test starts. Run it long enough to cover normal buying cycles, and avoid major promotions or stock problems that hit only one group. Change one thing at a time. And accept that some tests will show little or no lift; that result is just as valuable, because it tells you where budget can be cut.

Testing is especially important for offsite retail media and retail DSP campaigns, where attributed ROAS tends to look lower and the real question is whether the activity is bringing in shoppers who wouldn't have bought otherwise.

6Building a Cross-Network Measurement Framework

Most brands buy across several networks, so the goal is one way of judging all of them. A workable framework has four parts:

• Common definitions: agree how you will treat attribution windows, view-through conversions and halo sales, and document each network's native settings next to your own.
• Goals by funnel stage: judge search campaigns mainly on efficiency, and upper-funnel campaigns on reach, new-to-brand share and measured lift.
• One reporting layer: bring every network's data into a single dashboard with consistent naming, so you aren't comparing exports by hand. Our reporting and analytics team builds these views.
• A testing calendar: plan regular incrementality tests across networks and tactics, and use the results to adjust how much weight you give each network's own numbers.

Over time, the test results let you set simple adjustment factors for each network and tactic, so daily platform reporting can still guide decisions without overstating impact. For the wider operating picture, see our guide to managing campaigns across several retail media networks.

Retailers building their own networks face the same question from the other side: advertisers expect clear, consistent reporting and a credible answer on incrementality. Our retail media network operations work covers the advertiser reporting that supports this.

7Common Measurement Mistakes to Avoid

If you aren't sure how much of your reported retail media return is incremental, a free retail media audit is a good place to start. We will review how each network measures results, where the numbers overlap, and which tests would give you a clearer answer. You can also explore our full retail media marketing services or get in touch with our team.