Skip to content

Catalog creative testing: how to compare design variants without losing control

A beginner-friendly guide to comparing two designs with a clear hypothesis, one variable, a fair sample and careful attribution.

J

Julia Villegas

· 9 min read · 285 views
Two visual variants for catalog creative testing on a bright studio background

Catalog creative testing means comparing two ad designs generated from your product feed to understand which one helps shoppers respond, without changing too many things at once. If you manage hundreds or thousands of products, you need a simple method: one hypothesis, one variable, a comparable sample and a cautious view of attribution.

What catalog creative testing means

A catalog creative is the image or video that appears in a dynamic product ad using data from your product feed: product photo, title, brand, current price, previous price, discount, availability or other attributes. Instead of designing one ad manually for every SKU, you define a template and generate one creative per product.

Testing catalog creative means comparing version A with version B to answer a specific question. For example: “Does showing a discount badge make sale products easier to understand than showing the current price only?” The goal is to measure a design change, not a mix of design, audience, budget, bidding, seasonality and promotion changes.

For beginners, four terms matter most:

  • Hypothesis: what you expect to happen and why. Not “let’s try another design”, but “if we highlight the discount, sale products may be easier to evaluate quickly”.
  • Variable: the single element you change between A and B. It could be a discount badge, the size of the price, the background, the product crop or the order of fields.
  • Sample: the group of products, audiences or campaigns you compare. It should be similar enough across variants for the result to be useful.
  • Attribution: the way an ad platform assigns conversions or sales to ads. Platform attribution is useful, but it is not the same as real business margin.

Your feed can support creative variants, but the experiment itself depends on the advertising channel. Meta, Google, TikTok, Pinterest and other platforms have their own delivery systems, learning behavior, policies and attribution models. That is why you should separate two jobs: preparing clean, consistent variants in the feed, and designing the test correctly inside the channel.

Catalog creative testing: a simple method

The safest way to start is small. You do not need a complex testing architecture for your first experiment, but you do need discipline.

Decision Good practice Common mistake
Hypothesis Ask one clear question Change the design “to see what happens”
Variable Change one visual element Change background, price, image and copy at once
Products Use a stable, comparable set Mix bestsellers, clearance and new arrivals without control
Channel Follow specs and policies Create attractive images that the channel cannot use well
Reading Review performance with product context Decide only from aggregated ROAS

Fictional example: a sneaker store

Imagine an online sneaker store with 4,000 products. The team wants to know whether discounted products perform better when the creative includes a visible discount badge.

A simple test could be framed like this:

  • Hypothesis: for discounted products, a visible offer badge makes the ad easier to understand and may improve qualified engagement.
  • Variable: the presence or absence of the discount badge. Everything else stays the same: product photo, background, price position, typography and format.
  • Sample: products that are in stock, discounted and commercially relevant for paid campaigns.
  • Channel: Meta catalog ads, using the formats and policies required by the platform.
  • Reading: compare platform metrics, but also check margin context, stock changes and products that sold out during the test.

The useful takeaway would not be “template B always wins”. It would be more precise: “For discounted products in certain categories, highlighting the discount appears to help shoppers respond; next we should test whether this holds in other product sets.” That distinction matters because a catalog is not one ad. It is a collection of products with different prices, images, availability, margins and purchase intent.

How to prepare creative variants without breaking the feed

Before you launch a test, review the catalog. Dynamic creative is only as reliable as the data behind it. If the price is wrong, the discount field is missing, the product image is poor or stock is outdated, a new design will not fix the core problem.

Use this preparation flow:

  1. Choose the product set. Start with a commercially meaningful group: sale products, new arrivals, bestsellers or products with enough margin for paid media. Avoid testing the entire catalog first.
  2. Check the required fields. Review title, image, current price, previous price if used, brand, availability and URL. If a field is unreliable, do not make it the hero of the creative.
  3. Define the base template. Keep the structure stable: visible product, readable title or brand, clear price and simple hierarchy.
  4. Create one variant with one difference. Add a discount badge, increase price size or change the product crop, but do not change several elements together.
  5. Adapt formats by placement. A 1:1 creative does not behave like a 9:16 creative. If you plan to use Stories, Reels or other vertical placements, design the vertical format intentionally.
  6. Review platform policies and feed specifications. Each channel may have different rules for images, overlaid text, restricted products or required attributes.

With the AdMotion Templates Editor, a template can use dynamic catalog fields and visibility rules. That means a discount label can appear only when the product has a valid discount, or only when it belongs to a specific test set. The editor supports 1:1, 4:5, 9:16 and 16:9 formats, so you can adapt creative to placements without redesigning every product manually.

Product images also deserve attention. Background removal can make catalog ads cleaner, but it should be reviewed by a person. Some products are difficult to cut out, and small visual details can matter for purchase decisions. If you need layout inspiration before defining your variants, browse the ad examples gallery and look at how different formats handle hierarchy, product scale and price visibility.

How to do it with AdMotion

When you work with a large catalog, the hard part is not creating two designs. The hard part is keeping both designs organized, fed by the same data and ready for the channel. A practical AdMotion workflow would look like this:

  1. Connect the store or product file. Start from Shopify, BigCommerce, Tiendanube or any product file or feed. The goal is to use one catalog source for the creative variants.
  2. Map the fields. Match source fields to what you need for the creative and the channel: image, title, price, brand, stock, category, URL and offer-related fields if they exist.
  3. Normalize with rules. Use rules to clean values, group categories or create conditions. For example, if your feed has current price and previous price, you can identify discounted products when the previous price is higher than the current price.
  4. Create test sets. Filter a stable set for the experiment: available products, valid image, chosen category and the required pricing fields.
  5. Design template A. Build the base version with the dynamic fields you need, such as photo, title, brand and price.
  6. Design template B. Keep the layout consistent and change only one variable, such as a discount badge controlled by a visibility rule.
  7. Generate the channel feed. With Export Channels, you can prepare feeds for Meta, Google Shopping and Dynamic Feed, TikTok, Pinterest, Snapchat, Criteo, Microsoft Advertising, Thribee, ChatGPT or a custom channel. The analyzer checks each feed against the channel specification.
  8. Copy the feed URL into the channel. The feed is published through a URL generated at each download. Remember that your catalog import frequency and the channel’s feed reading frequency are different; you should not assume instant synchronization inside the ad platform.

This workflow does not replace the experiment setup inside the ad channel. It helps you prepare consistent creative variants, but delivery, comparison and final attribution depend on how you configure the campaign and how the platform processes the data.

For feed preparation, the AdMotion Feed Manager is the part that connects the store or file, maps fields, cleans and segments the catalog with rules, and publishes a feed URL for each channel. For creative production, the Templates Editor turns those fields into one image per product. These are preparation tools; they do not guarantee approval, delivery or performance in any advertising platform.

How to read results without fooling yourself

The biggest mistake is looking at one metric and declaring a winner too quickly. In catalog campaigns, one variant can look better because it received more popular products, because it overlapped with a promotion, because stock was healthier, or because the channel delivered impressions differently.

Separate the result into three layers:

  • Ad metrics: impressions, clicks, CTR, attributed conversions, cost and platform ROAS.
  • Catalog data: price, category, stock, discount, product age and margin context if you track it outside the ad platform.
  • Campaign conditions: budget, audience, placement, timing, learning phase and any changes made during the test.

If you run Meta campaigns, Meta Analytics can combine Meta Ads metrics with catalog data product by product and export reports. It is available on request with read-only permissions. That matters: it is for analysis, not for pausing campaigns, changing bids or modifying budgets.

You should also distinguish attribution from real margin. Attribution tells you which conversions a platform assigns to an ad according to its own rules. Real margin depends on product cost, discounts, returns, logistics, payment fees and other business data that may not live inside the ad platform. A creative that sells more low-margin products may not be the best business decision, even if its platform ROAS looks attractive.

A beginner-friendly way to avoid overreacting is to write down what would count as a useful result before you launch. For example: “We will not decide from ROAS alone. We will compare the variant by product set, exclude products that went out of stock during the test, and check whether the winning signal is concentrated in one category.”

Common mistakes when comparing catalog designs

Most early catalog creative tests fail for predictable reasons. Watch for these:

  1. Changing too many things at once. If variant B has a different background, price size, product crop, badge and format, you will not know which change mattered.
  2. Testing unstable products. If part of the set goes out of stock during the test, the comparison becomes weaker.
  3. Ignoring format. A design built for 1:1 may not work in 9:16. Plan by placement, especially for vertical environments.
  4. Mixing categories without context. A discount badge may help in fashion but matter less in technical products where shoppers compare specifications.
  5. Ending the test too early. Channels need enough delivery and time to produce a meaningful signal. Do not make decisions from a short fluctuation.
  6. Forgetting channel policies. You may choose different images for each channel, but those images still need to respect each platform’s specifications and advertising policies.

The solution is not to stop testing. It is to document every test like a short experiment brief: hypothesis, variable, product set, dates, channel, result and decision.

Final check before you launch

Use this checklist before you push a catalog creative test live:

  • The hypothesis is written in one sentence.
  • Only one visual variable changes between A and B.
  • The product set is stable, available and commercially meaningful.
  • Required fields are clean enough to appear in the creative.
  • Formats match the placements you plan to use.
  • The feed has been checked against the channel specification.
  • You know which campaign metrics and catalog fields you will review.
  • You have agreed not to treat platform attribution as the same thing as real margin.

If one of these items is missing, fix it before launching. A slower, cleaner test is usually more useful than a fast test that no one can interpret.

What to do this week

  1. Pick one business question. For example: “Should we highlight discounts on sale products?” or “Does a larger price help in fast-decision categories?”
  2. Choose a small, stable product set. Use products with stock, valid images and reliable price data. Do not start with the full catalog.
  3. Design two variants with one difference. Keep image, format, hierarchy and fields the same except for the variable you want to measure.
  4. Prepare formats by channel. Use 9:16 when you need vertical placements, and review 1:1 or 4:5 for feed and carousel-style environments where relevant.
  5. Define how you will read the result. Decide the metrics, review period and catalog fields before launch, so you do not confuse attribution with profitability.

If you want visual ideas before building your own variants, start with examples by format and vertical. And if the challenge is producing consistent variants for many products without designing each one by hand, the Templates Editor can help turn catalog fields into reusable creative systems.

Frequently asked questions

How many variables should I change in a catalog creative test?

Change one variable at a time whenever possible: a badge, price size, background, product crop or field order. If you change several elements together, you may see a performance difference, but you will not know which design choice caused it. That makes the learning hard to reuse.

Can I use the same catalog creative design across every channel?

You can start from the same creative idea, but you should adapt formats and images to each channel. A 1:1 design will not behave like a 9:16 design, and platforms have their own specifications and policies. Consistency is useful, but it should not replace channel-specific preparation.

Does the product feed decide which creative variant wins?

No. The feed can contain the fields, rules and variants needed to generate creatives, but the actual experiment depends on the advertising channel and its delivery. To make a better decision, combine campaign metrics with catalog data such as stock, price, category and margin context if available.

Sources

  1. Editor de Plantillas de AdMotion — AdMotion
  2. Feed Manager de AdMotion — AdMotion
  3. Meta Analytics de AdMotion — AdMotion
Share:
J

Written by

Julia Villegas

Performance Marketing

Julia Villegas works in performance marketing at AdMotion.

Related posts