Augmented reality in beauty: what virtual try-on does, and who actually builds it

Beauty was the first retail category where augmented reality stopped being a marketing stunt and became a normal part of the product page. If you sell lipstick online, a shopper can point a phone at their own face and see the shade on their own lips before adding it to the basket. That is a real answer to a real question, and it is why the beauty industry adopted AR years before furniture did.

We should say up front that we do not build face try-on. CharpstAR makes 3D models and web AR for physical products with a fixed shape, furniture, stoves, home goods and similar. Face tracking for cosmetics is a different discipline with different vendors. This article is written for the person in a beauty business who has been asked to evaluate it and wants a straight description rather than a pitch deck.

What AR try-on for cosmetics actually is

Two pieces of technology sit behind every makeup try-on. The first is face tracking: the phone or browser finds the face in the camera image and works out where the lips, eyelids, cheeks and brows are, several dozen times a second. Modern phones do this well, and it is largely a solved problem.

The second is rendering the product on top. This is where the difficulty lives. A lipstick is not a flat colour patch. It has opacity, gloss, a little translucency where the natural lip colour shows through, and it behaves differently on a dry lip than a full one. Foundation is harder again, because the whole point is matching a tone to skin that the camera is already misreading under whatever light the shopper is standing in.

So the accuracy of a try-on varies enormously by product type. Shopify's guide to AR shopping reports difficulty scores by category, with facial complexion products the hardest at 41 percent, followed by self-tanning and body makeup at 38 percent each. That ranking matches what you would expect: the closer the product is to reproducing skin itself, the worse the result.

What it does well

Colour choice within a known family. Lipstick, eyeshadow, blush, hair colour, nail polish. The shopper is not asking "is this my exact shade", they are asking "does this red work on me or is it too orange", and a try-on answers that in a second.

Discovery and play. The engagement numbers are real. L'Oréal, which owns ModiFace and runs try-on across many of its brands, reported over 100 million digital try-on sessions in 2023 against 40 million in 2022. That is not a conversion statistic, it is an attention statistic, and it tells you shoppers pick the feature up voluntarily rather than needing to be pushed into it.

The commercial claims are softer than the vendor decks suggest. L'Oréal's then chief digital officer Lubomira Rochet told The Drum that putting AR on a site doubled engagement time and tripled conversion rate. That is a strong number from a credible source, but it is a brand-side statement without a published methodology, and it compares shoppers who used the feature with those who did not, which always flatters the feature. Treat it as directional.

Where it still fails

Foundation and concealer shade matching is the honest weak point. The camera white-balances to the room, the screen has its own colour profile, and the shopper is standing under a kitchen bulb. Vendors mitigate this with reference cards, controlled-lighting prompts and laboratory-measured shade data, and the best of them are good, but nobody has removed the underlying problem. If your returns are driven by wrong-shade foundation, a try-on will help less than a well-run sample programme.

Texture claims fail too. A try-on cannot tell a shopper whether a mascara clumps, whether a cream feels heavy, or whether a shade shifts three hours into wear. Those are the things reviews are for, and a try-on does not replace them.

Who provides it

The market is concentrated in a handful of specialists rather than something you build in-house.

Two practical notes. First, per-shade setup work is the real cost, not the licence. A 400-shade catalogue has to be measured and tuned, and that effort scales with the catalogue, not with the app price. Second, check whether the try-on runs in the mobile browser or only inside an app or a social platform, because a shopper who has to install something will not.

How to judge a demo

Vendors demo on flattering faces in flattering light. Do this instead.

• Try it on at least four skin tones, including the darkest and lightest you sell to. Uneven performance across tones is common and is the single biggest reason a launch goes badly.
• Test in bad light. Evening, a yellow bulb, a window behind you.
• Compare against the real product side by side, on a real face, photographed together.
• Check load time on a mid-range Android phone over mobile data, not on the demo iPhone.
• Ask what happens on a desktop browser with no webcam, because a meaningful share of your traffic is there.

What transfers to the rest of your catalogue

Beauty brands sell more than face products, and the parts of the range with a fixed shape behave like any other physical product. Packaging, tools, devices, salon furniture and retail fixtures can be modelled once in 3D and placed at true scale in a room or shown from every angle on the page. That is the work we do, and it is a different pipeline from face tracking: a model is built, hosted, dropped into a viewer and reused for renders and AR.

It is worth knowing what that costs, because it is a fraction of a try-on programme. Our own plans run at 10 dollars per product per month on the basic tier and 20 dollars on the tier that adds dimensions, annotations and a colour configurator, with details on the prices page. Companies like MELIMELI use that configurator so shoppers can switch fabric on a live model, which is the same idea as a shade switcher applied to an object instead of a face.

The short version

AR try-on in beauty is genuinely useful for colour products, genuinely limited for complexion matching, and mature enough that you should be buying it from a specialist rather than building it. Judge it on diverse faces in poor light, budget for shade setup rather than licence fees, and keep reviews and samples in place for everything AR cannot show.

If the fixed-shape part of your catalogue is still living on photographs, that is the piece we can help with. Send us one product and we will model it and put it in a viewer for free, yours to keep either way: free sample. More on what we build is on the solutions page.

Sources

• Shopify, AR shopping: category difficulty scores, Rebecca Minkoff and Gunner Kennels results, try-on app pricing.
• Shopify, virtual fitting rooms: providers and pricing.
• PYMNTS, L'Oréal sees 150 percent increase in virtual try-ons.
• The Drum, Conversion rates triple when L'Oréal uses AR tech to showcase products.