Fashion6 min read
How to run an AR pilot in retail and measure it properly
A practical method for testing AR on product pages: what to pilot, how to split traffic, which metrics to record, and how to read the result honestly.
By CharpstAR · 4 Oct 2023 · Updated 22 Sept 2026
Search for an AR retail case study and you will find a lot of pages with confident percentages and no methodology. Some of them are real. Many compare shoppers who chose to use a feature against shoppers who did not, which measures interest rather than effect, and a few appear to have been invented outright. This article is the thing those pages should have been: a method for running your own pilot so you end up with a number you can defend internally.
We build 3D and AR for retailers, so we have an obvious interest in the answer. That is exactly why the method below is designed to be able to tell you no.
Why published figures are not enough
The best-known numbers are worth knowing. Shopify reported in September 2020 that interactions with products carrying 3D or AR content converted 94 percent more often than comparable products without. Rebecca Minkoff, in Shopify's case-study write-up, found shoppers 44 percent more likely to add an item to cart after viewing it in 3D and 65 percent more likely to purchase after viewing it in AR. Gunner Kennels, in the same source, cut return rates by 5 percent.
Two of those three are interaction-based comparisons. A shopper who opens the 3D viewer has already leaned in. Part of the measured lift is that leaning in, not the viewer causing it. This does not make the numbers fake, it makes them an upper bound. Your pilot exists to find your number.
The other reason to run your own is that the size of the prize varies enormously by category. The National Retail Federation and Appriss Retail put United States retail returns at 743 billion dollars in 2023, 14.5 percent of sales, with online returns at 17.6 percent against 10.02 percent in store. If your category returns at 5 percent, a returns-driven business case will not stand up. If it returns at 25 percent, it might carry the whole project.
Step one: pick the right products
Choose products where the shopper's unanswered question is a visual or spatial one. High price, high consideration, and a real question about size, space, finish or configuration. Furniture, appliances, large home goods, eyewear, anything modular.
Pick enough of them. This is where most pilots die. Three products will not produce a readable result, because conversion rate on a handful of pages is mostly noise. You want a set of products that between them see a few thousand sessions a month over the pilot period. If your traffic is thin, extend the pilot rather than shrinking the product set.
And pick a control set that looks like the test set: similar price band, similar category, similar traffic, running at the same time.
Step two: decide how you split
There are three designs, in descending order of quality.
A true A/B split is best: the same products, half the traffic sees the 3D and AR view, half does not, randomised at the visitor level and held for the whole pilot. If your platform or testing tool supports this, use it and stop reading this section.
A matched-product comparison is next: test products carry the feature, control products do not, both running simultaneously. Weaker, because product-level differences leak into the result, but workable if the sets are genuinely comparable.
A before-and-after on the same products is the fallback: measure for a month, launch, measure for a month. This is the weakest design because seasonality, pricing, stock and marketing all move underneath you. If it is all you have, run it over a stable period, note every other change that happened, and read the result as an indication rather than proof.
Step three: record these metrics
| Metric | Why it matters | How to read it |
|---|---|---|
| Conversion rate, test against control | The headline business case | The only number your finance team will care about |
| Return rate, test against control | Usually the larger financial effect | Needs a longer window than conversion, at least one return cycle |
| Viewer engagement rate | Share of sessions that open the 3D view | Diagnoses placement, not value. Below 10 percent means the viewer is hidden |
| AR launch rate on mobile | Share of mobile sessions tapping into AR | Tells you whether the AR half is being used at all |
| Time on product page | Directional signal of engagement | Never use alone; longer can mean confused |
| Add-to-cart rate | Earlier signal than conversion, less noisy | Useful when traffic is too low for conversion significance |
| Support contacts per order | Often the fastest visible win | Size and fit questions usually drop first |
Record them before you launch as well as after. A pilot with no baseline is not a pilot.
Step four: run it long enough
A month is the minimum for conversion. Returns need longer, because a customer who bought in week one may not return until week six, so run the returns measurement for at least one full return window past the end of the pilot. Do not stop early because the first week looks good. Early stopping on a favourable swing is the most common way a pilot produces a number that does not survive rollout.
Keep a change log. Every price change, every campaign, every stock-out on a test product goes in it. You will need it when someone asks why week three looks odd.
Step five: read it honestly
Three traps.
Novelty. Engagement with a new feature is inflated for the first few weeks. If your pilot is short, you measured the novelty.
Self-selection. If you report only on shoppers who used the viewer, you are reporting on interested shoppers. Report on all traffic in the test group, whether or not they touched the feature. That number is lower and it is the true one.
Placement confusion. A low engagement rate usually means the viewer is below the fold or behind a tab, not that shoppers do not want it. Fix the placement and rerun before concluding the feature failed.
What a result looks like
At the end you should be able to say something like: across 40 products and two months, the test group converted at a stated rate against a stated control rate, returns ran at a stated difference, 18 percent of sessions opened the 3D view and 6 percent of mobile sessions launched AR, and support contacts on those products fell by a stated amount. Then multiply the conversion and returns difference across the catalogue you would roll out to, set it against the monthly cost per product, and you have a decision rather than an opinion.
Our own pricing is deliberately shaped for this, at a fixed monthly fee per product on the prices page, because a per-product monthly cost is easy to hold against a per-product measured gain.
The clients we point to did exactly this kind of narrowing before scaling. Sweef started with the configurable sofa where the questions were worst, MELIMELI with the fabric range that photography could not cover, and Contura with stove placement. None of them started with the whole catalogue.
If you want to run this experiment, the first product costs nothing. Send us one and we will model it and put it in a live viewer, yours to keep whatever the pilot concludes: free sample. More detail on what we build is on the solutions page.
Sources
- Shopify, merchant data on 3D and AR content, September 2020.
- Shopify, AR shopping: Rebecca Minkoff and Gunner Kennels results, and 3D ecommerce guide.
- National Retail Federation and Appriss Retail, 2023 Consumer Returns in the Retail Industry.
