Product colour inspection with a camera, measured inside the mask

The carton sleeve for a skincare range is a particular teal, and the brand cares about it more than about anything else on the pack. On the fourth print run of the Tuesday shift the sleeves coming off press 2 are a shade greyer, and the operator does not see it because the operator has been looking at that teal for six hours. The customer's buyer sees it on the shelf next to last month's stock and sends the lot back.

A camera over the conveyor after the folder-gluer can compare every sleeve with the reference. Whether that comparison means anything depends on three decisions made before the first sleeve goes past.

The product is isolated by a mask so the belt stays out

A box around the sleeve is the wrong tool for a colour measurement, because a box is a rectangle and a sleeve on a conveyor is not. The corners of the box hold the belt, and the belt on line 3 is blue. Any colour statistic taken over the box is a blend of teal and belt in a ratio that changes with how the sleeve landed.

So the sleeve gets a mask rather than a box: a polygon traced around its outline, with the pixels inside it the only ones that count. You type "carton sleeve" once, Lexi proposes the polygon on every frame, and a person checks the edges before anything trains. On the polygon edges the checker is looking for belt pixels inside the line and sleeve pixels outside it, since either one pollutes the number.

A fabric line has the same problem with a different belt. The roll edge on a dyeing line is measured inside a mask that stops at the selvedge, because the selvedge is white and the pass or fail is about the cloth.

Colour is quantised to a few swatches rather than averaged

With the mask in place on line 3, the naive step is to average every pixel inside it. The average of a teal sleeve with white product text on it is a pale teal that appears nowhere on the pack, and a sleeve whose teal is right and whose text is smudged reads the same as a sleeve whose teal is off.

The better step is to quantise: cluster the pixels inside the mask into a handful of dominant colours, with the share of the mask each one covers. A sleeve then comes back as three swatches, the teal at most of the area, the white text at a small share, and a gold foil accent at less. Each swatch is compared on its own. The teal is what the brand cares about, and the teal now has its own number rather than being dragged toward white by the text.

A word on the colour space. A difference that looks small in the raw camera values can be large to an eye, and the reverse, so the comparison is made in a perceptual space where the distance between two swatches roughly matches how different a person would call them. The tolerance the brand sets is a distance in that space.

The reference is measured on the same camera

The brand supplies a reference as a printed swatch and a set of values from a spectrophotometer. Neither is what the camera sees. The camera over line 3 has its own sensor, its own white balance, and the plant's own lights above it, and a sleeve that matches the brand file perfectly will not produce the brand file's numbers through that camera.

The reference for the camera is a sleeve the brand has approved, placed on the belt under the camera, and measured the same way as every sleeve that follows. The tolerance is then a distance from that measurement, not from the brand file. My own preference is to keep the approved sleeve physically at the line, in a drawer at the press 2 station, and to re-measure it at the start of every shift, because the lights over the belt age and the measurement drifts with them.

The press operator on line 3 still holds a sleeve up to the window when the light is right. That check is slower than the camera and it is still the one the operator trusts, which is fine, because the two agree most days and the days they do not are the interesting ones.

A pass or fail on the line has to be quick and quiet

The package and label inspection use case sets the constraint that shapes everything above: line speed. The decision is made in the gap between sleeves, on frames that are sometimes blurred, and a false reject stops the line just as expensively as a false accept sends the grey lot to the customer.

So the rule that fires is written to be slow to reject. A sleeve whose teal is outside tolerance on one frame is not a reject; a run of sleeves outside tolerance is. The alert is a sentence: teal on line 3 outside tolerance for a run of sleeves, high severity, with a cooldown, approved before it goes live, and what arrives is the frame with the mask drawn and the measured swatch beside the reference swatch. The operator at press 2 looks at both and adjusts the press before the run is lost.

A swapped camera moves every measurement on the same day

The failure nobody plans for is maintenance. The camera over line 3 fails in month eight and is replaced with the same model from the spares cupboard, or a newer one because the old model is discontinued. From the moment it is powered on, every sleeve measures a shade different. The mask still finds the sleeve, the quantiser still finds three swatches, and the teal is now outside tolerance on every frame, with nothing wrong at the press.

The drift catalog files this as a camera that was replaced: new hardware with a different colour response and sharpness, and the model treats a familiar scene as unfamiliar. For a colour job the effect is worse than for a detection job, because colour is the whole measurement. The order of repair is the reference first, the model second. Re-measure the approved sleeve on the new camera, reset the tolerance around that, and only then look at whether the mask edges still hold, which they usually do.

The doubted frames are the ones in the shadow of the guard

LexData takes the sleeve model through its whole life. You type what to look for, Lexi puts a polygon on every sleeve in every frame, and a person checks each label before anything trains on it. The model then watches the camera over line 3, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The frames that come back are a sleeve half under the shadow of the conveyor guard, two sleeves overlapping after a jam, and the foil accent catching the light and reading as white. Each one is a mask corrected by a person, and when the corrections cross the project's threshold a new version trains.

The colour comparison sits downstream of the mask and does not change when the mask model does, which is the reason to keep them apart.