Coffee bean inspection with computer vision on the roasted bean belt

After the cooling tray, roasted beans move along the belt on line 2 in a single, shifting layer towards the hopper. Somewhere in the stream is a small dark stone that came in with the green lot, survived the destoner because it weighs about what a bean weighs, and is now the same colour as a dark roast. If it reaches a customer's grinder it wrecks the burrs. An air jet a metre down the belt can knock it off, but only if something tells the jet exactly where the stone is at the moment it passes.

That last clause is the whole design.

The camera does not need to know that the stream is coffee. It needs to give the jet a position, and the position has to be right, on a belt that moves.

A bounding box is too coarse for the air jet to aim at

A bounding box around a stone half hidden by two beans covers the stone and a fair amount of bean. Fire the jet at the centre of that box and it may hit a bean and miss the stone, or blow three good beans off line 2 for one pebble. On a dense stream the box is too coarse an answer.

A mask is the right shape. Instance segmentation draws the outline of each stone, even where it is partly covered, and the jet fires at the centre of the outline. Two stones touching get two masks instead of one shared box. The labeling costs more, since a polygon takes longer than a box, and my view is that the polygon earns its cost on any belt with a reject jet and rarely on a belt that only counts.

Stones, quakers and foreign material are defined before labeling

The classes come before the frames. Stones are the dangerous one, and they arrive in several forms: pale ones that stand out against a dark roast, dark ones that vanish into it, small pebbles the size of a broken bean. Twigs, husk and the occasional fragment of something metal belong with them, so the class on line 2 is foreign material, meaning anything on the belt that is not a bean.

Quakers are the harder class because they are beans. An underdeveloped bean roasts pale and tastes of paper, and it has the same shape and size as its neighbours, so the model has to learn it from colour and surface rather than outline. Roasters call them quakers, and nobody at a roastery seems able to say with certainty where the name came from.

Each class gets a written rule before labeling starts: how small a stone still counts, whether a stone half under a bean is outlined to its visible edge, and what the labeler does with a bean that looks like a stone. Without those, two labelers produce two datasets under one class name.

The frames come from the belt and never from the bench

The commonest way to fail is to train on beans arranged on a white sheet under a lamp. The belt is dark, the beans overlap, the LED strip over the camera throws one kind of light and the night shift's ambient throws another, and the belt moves fast enough to blur. A model trained on the bench fails on the belt on its first day.

So the frames come from the mounted camera at its height, under the line's own light, at the line's own speed, across the roast levels the plant actually runs. Frames with no defect at all go in too, so the model has seen a clean stream and finding nothing is an answer it has given before. You type the classes once, Lexi proposes the outlines on every frame, and a person checks each one before anything trains on it, which is where the half-hidden stones and the missed quakers get caught.

A new origin lot moves the defect rate without breaking the model

The model has been running well for two months when a new origin lot arrives in October, harvested somewhere stonier. The frames have not changed in kind, and the model is still finding stones correctly. What changed is how many there are. The jet fires constantly, the review queue floods, and the line lead's first instinct is that the model has broken.

It has not. The drift catalog calls this a change in the defect rate: the inputs held, the definitions held, and the base rate moved. Per-detection accuracy is the same. Alert volume against its own history is the cheapest place to see it, and it is a number the plant already has. When the rate moves because the world moved, that is a finding about the lot, and it is worth telling procurement before touching the model.

Thresholds are re-tuned before the model is blamed

Every threshold on the line was tuned around the rate before the October lot, and when the rate moves, the thresholds are wrong before the model is. On the food lines we run, the first move is to re-tune against the current base rate and see whether the queue calms down. Only if it does not, and the corrections at review start to show a real pattern, is the model the thing to retrain.

The confidence lesson covers why a threshold is an operating point against the real frequency of the thing you are hunting, and why a rare defect keeps generating false alarms even from a model that is right most of the time.

Corrections at review are the signal the model needs retraining

When the person reviewing the queue starts overriding the model in one direction, stones passed as beans or a new kind of pebble missed, that override rate is what says the model has something to learn. Corrections at review retrain it. Asking a question about the footage does not.

LexData takes the bean model through its whole life. You type what to look for, Lexi outlines every stone and quaker on every frame, and a person checks each label before anything trains on it. The model then watches the belt camera, on a runner beside the recorder so the footage stays on site, or on your servers. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The next origin lot, the next roast profile and the lamp that ages over a summer all land as corrections, and the model that was tuned in the spring is still right in the autumn.

The surface defect use case is the general form of this line. Disposition is a severity call, severity is size and location, and only a mask carries both. On a bean belt the disposition is a puff of air at a point, and the mask is what makes the point right.