Recognising fabric from a photograph: what AI gets right, and what it misses

Recognising fabric from a photograph belongs to that class of problems that look trivial until you attempt them. A professional identifies a fabric in a second, through touch and glance. Reproducing that judgement from a single still image is markedly harder.

What works well

Today’s multimodal vision models are reliable on anything with a strong visual signature:

• Garment type — shirt, jacket, coat, dress, trousers: near-systematic identification.
• Fabrics with characteristic texture — denim, leather, heavy knit, velvet, fur: the weave reads on the image.
• Visible stains — presence and location are detected consistently.
• At-risk trims — mother-of-pearl buttons, sequins, applied leather, glued ornaments: well spotted once resolution allows.

On these categories, performance exceeds that of a rushed operator — not because the model “sees” better, but because it never tires and applies the same criterion to the thousandth garment as to the first.

What stays hard

The confusions are not random: they follow precise families.

Convincing imitations

A well-woven viscose imitates silk closely enough to fool a human eye on a photograph. Drape, the one discriminating clue, does not read reliably on a garment laid flat. The same difficulty arises between certain wool-polyester blends and pure wool.

Blends

A fabric of 60 % cotton and 40 % polyester has no appearance of its own: it looks like whichever of the two dominates visually, which is not necessarily the one that dominates by proportion. Yet it is the polyester that constrains treatment temperature.

Shooting conditions

Yellow counter lighting shifts hues; a flash flattens texture; a creased garment changes how the weave reads. A significant share of errors comes from the image, not the model.

Why confidence matters more than the prediction

A model that is wrong without knowing it is dangerous. A model that is wrong while flagging its uncertainty is usable. All the practical value lies in that difference.

So we always expose a confidence score, and we calibrate it to be honest: across all predictions announced at 80 %, roughly 80 % should be correct. An overconfident model is worse than a cautious one, because it teaches the operator to stop checking.

The right question is not “is the model right?” but “does it know when it is likely to be wrong?”

Predicting what serves, not what impresses

One choice did a great deal for the reliability of our results: we do not try to predict an exact composition. We predict a treatment path.

Telling viscose from silk is hard. Determining that the garment belongs to the “delicate fibres, gentle wash, no heat” family is far more robust — and that is the information the workshop actually uses. So we grouped fabrics by treatment constraint rather than by textile nomenclature.

The gain is twofold: residual errors occur within a single family, and therefore have no consequence on the treatment applied. And the operator receives an actionable instruction rather than a label to interpret.

What we correct continuously

Every operator correction is recorded with the photograph and its context. Those gaps are reviewed periodically: they show where the model’s instructions fall short, which thresholds sit wrong, and which fabric families need special handling.

It is slow work with no spectacular effect. It is also the only kind that durably improves a system deployed in the field.